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FOREWORD 


The  Workshop  on  Adaptive  Control  was  held  to  assess  the  state  of  the  art  In 
adaptive  control  research.  It  was  felt  that  with  the  recent  advances  In 
control  theory  research  along  with  the  revolution  In  microprocessor  capabil- 
ities that  real  adaptive  control  systems  may  be  a possibility  in  the  near  fu- 
ture. A second  objective  ( although  not  In  sequential  order)  of  the  workshop 
was  to  identify  future  research  topics.  The  right  mixture  of  practitioners 
and  theoreticians  provided  the  right  catalyst  to  answer  both  objectives. 

This  copy  of  the  final  report  represents  the  results  of  that  workshop. 


I wish  to  express  special  thanks  to  Dr.  J.B.  Cruz,  Jr.  of  Dynamic  Systems  for 
organizing  the  workshop.  However,  his  excellent  job  would  not  have  been  evi- 
dent without  the  involvement  of  the  participants.  To  the  participants  I say 
thanks  for  a job  well  done. 


CHARLES  L.  NEFZGER,  Major,  USAF 
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I.  INTRODUCTION 

A.  Background 

Recent  advances  in  adaptive  control  theory,  stochastic  control  theory, 
computational  algorithms  for  control,  and  computer  technology  point  towards 
the  feasibility  of  controllers  for  navigation,  guidance,  and  flight  control 
systems  which  function  over  a wide  range  of  adverse  operating  conditions. 

Furthermore,  recent  results  indicate  that  fixed  structures  with  no  adaptation 
are  capable  of  operating  over  a wide  range  of  conditions.  One  purpose  of  the 
workshop  was  to  provide  a forum  for  assessing  the  state  of  the  art  and  for 
relating  recent  theory  to  current  control  problems  in  aerospace  technology 
and  other  fields.  A second  purpose  of  the  workshop  was  to  have  an  open 
discussion  of  future  directions  in  research  on  adaptive  control,  stimulated 
in  part  by  projected  needs. 

The  field  of  control  exists  because  of  the  need  to  operate  a system 
satisfactorily  in  spite  of  uncertainty  associated  with  the  dynamic  system  to 
be  controlled.  The  uncertainty  comes  about  because  of  inaccuracies  in 
modeling,  changes  in  environmental  conditions,  and  presence  of  unavoidable 
disturbance  inputs.  The  cornerstone  of  control  theory  is  feedback  theory, 
which  originated  with  electronic  amplifiers  [1-3].  The  first  books  on  control 
systems  [4,5]  and  several  dozen  others  written  in  the  last  30  years  rely  on  . j ' 

the  potential  benefits  of  feedback  to  counteract  the  effects  of  uncertainty. 

With  increased  complexity,  multiplicity  of  feedback  loops,  and  large  parameter 
variations,  the  exploitation  of  feedback  is  not  easy  but  still  possible  [6]. 

In  addition  to  stabilization,  reduction  of  sensitivity  to  parameters,  reduction 
of  nonlinear  distortion,  and  reduction  of  effects  of  disturbance  inputs, 
feedback  may  be  useful  for  maintaining  near-optimality  for  a range  of  para- 
meters [7]. 
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When  the  changes  in  parameters  are  large  or  when  there  are  extreme 
variations  in  environmental  conditions  such  as  in  aircraft  control  over  a 
wide  range  of  flight  conditions,  a fixed  control  may  not  be  adequate.  Start- 
ing in  the  mid  fifties  self-adaptive  controls  whose  control  parameters  change 
in  consonance  with  changes  in  the  controlled  process  have  been  proposed 
[8,9].  In  the  last  twenty  years  there  have  been  significant  advances  in 
adaptive  control. 

Feedback  systems  with  fixed  controllers  were  called  passive  adaptive 
systems  [8]  in  the  early  days  of  adaptive  control.  Many  advances  have 
occurred  in  passive  adaptive  systems  and  there  is  a resurgence  of  interest  in 
this  subject.  The  present-day  terminology  for  this  approach  is  robust  control. 
Since  this  is  a simpler  control  structure,  it  is  an  attractive  alternative  to 
active  adaptive  control  whenever  the  control  problem  could  be  solved  using  a 
robust  control. 

In  much  of  robust  control  and  adaptive  control,  the  uncertainty  is 
described  in  terms  of  deterministic  concepts.  Alternatively,  these  parameter 
uncertainties  could  be  modeled  probabilistically.  Furthermore,  the  disturbance 
inputs  could  be  modeled  as  stochastic  processes.  Such  an  approach  leads  to 
the  theory  of  stochastic  control  [10,11].  A special  class  of  stochastic 
control  problems  has  been  successfully  solved  using  the  method  of  self-tuning 
regulators  [12].  The  more  general  problem  is  much  more  complex  and  thus  far, 
the  available  results  are  at  a conceptual  and  theoretical  level  [13].  There 
is  much  current  work  on  obtaining  simpler  algorithms  and  perhaps  in  a few 
years,  there  would  be  some  substantial  applications. 
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B.  Organization  of  Workshop 

The  workshop  concentrated  on  three  topics:  a)  robust  control,  b)  model 
reference  adaptive  control  and  self-tuning  regulators,  and  c)  stochastic 
adaptive  control.  Although  self-tuning  regulators  are  special  stochastic 
adaptive  controls,  they  were  lumped  with  model  reference  adaptive  controls 
because  of  recent  results  unifying  these  two  areas.  There  were  thirty-four 
participants  from  industry,  universities,  and  government  laboratories.  The 
first  session  was  devoted  to  a presentation  of  three  key  papers,  one  from 
each  of  the  discussion  topics.  The  next  session  was  devoted  to  a general 
discussion  of  the  three  topics.  The  second  day  was  devoted  to  separate  dis- 
cussions of  the  three  topics.  The  workshop  participants  were  divided  into 
three  working  groups  with  a discussion  leader  for  each.  The  last  session  on 
the  third  day  was  devoted  to  a presentation  of  the  conclusions  of  the  three 
groups . 

All  participants  were  urged  to  prepare  short  statements  on  their 
starting  points  for  the  discussions.  The  submitted  Individual  contributions, 
as  well  as  the  three  key  papers,  are  included  in  this  report.  The  working 
groups  were  asked  to  examine  the  state  of  the  art  of  the  subfields,  to  examine 
application  areas  where  the  present  results  might  be  used,  and  to  explore 
future  research  directions.  The  three  working  groups  were: 

Working  Group  on  Robust  Control 

J.  Ackermann,  Discussion  Leader  R.  K.  Mehra 

D.  Bowser  C.  L.  Nefzger 

D.  P.  Looze  W.  R.  Perkins 

R.  G.  Marsh  M.  G.  Safonov 

J.  Medanlc  K.  K.  D.  Young 
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Working  Group  on  Model  Reference  Adaptive  Control 


and  Self-Tuning  Regulators 


I.  D.  Landau,  Discussion  Leader 

G.  F.  Franklin 

C.  A.  Harvey 
C.  R.  Johnson 

H.  Kaufman 

P.  V.  Kokotovi<^ 


G.  Kreisselmeier 
L.  Ljung 
R.  V.  Monopoli 
A.  S.  Morse 
K.  S.  Narenda 
E.  G.  Rynaski 


Working  Group  on  Stochastic  Adaptive  Control 


Y.  Bar-Shalom,  Discussion  Leader  C.  S.  Padilla 


P.  E.  Caines 
J ■ 9*  Cruz,  Jr* 
J.  Dillow 
B.  Friedland 

D.  G.  Lainiotis 


T.  Riggs 
A.  V.  Sebald 

E.  C.  lacker 
L.  Tesfatsion 
P.  Vergez 
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ROBUST  CONTROL 


Juergen  Ackermann 
Coordinated  Science  Laboratory 
University  of  Illinois,  Urbana,  111.  61801 
on  leave  from  DFVLR  Oberpfaf fenhofen 


1.  Robust  Control  Problems 

Robustness  of  control  systems  is  defined  in  terms  of  system  properties, 
which  are  invariant  under  a specified  class  of  perturbations.  Typical 
examples  of  desirable  system  properties  are: 

1)  Stability  or  nice  stability  (e.g.  defined  by  constraints  on  eigen- 
value locations). 

2)  Limited  deterioration  of  a performance  index. 

3)  Limited  deviation  from  an  ideal  behavior,  e.g.  constraints  on  step 
responses  or  frequency  responses  or  on  the  return  difference. 

4)  Limited  deviation  from  a reference  behavior,  e.g.  deviation  from  a 
nominal  trajectory  or  a reference  model  response. 

5)  Tracking,  i.e.  zero  asymptotic  error  for  a class  of  reference  and 
disturbance  inputs. 

6)  Limited  demand  on  control  |u  | and  control  rate  |u|. 

Perturbations  may  occur  in  the  structure  or  in  the  parameters  of  a system. 
Examples  of  structural  perburbations  are: 

1)  Sensor  failures. 

2)  Actuator  failures. 

3)  Switching  from  automatic  to  manual  control.  Here  it  is  desirable 

Cr 

that  the  operator  sees  a stable  system  whenever  he  opens  one  more 
feedback  loops. 

4)  Change  in  system  order  due  to  a failure.  Example:  An  aggregate 
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description  for  several  power  generators  or  a traffic  flow  or 
economic  variables  must  be  dissolved  into  a more  detailed  description 
of  transients  between  individual  components  in  failure  situations. 
Parametric  perturbations  are  due  to  uncertainties  in  the  plant  model  and  in 
the  controller  Implementation.  Examples  are: 

5)  Analytically  known  dependence  of  a plant  model  on  uncertain  physical 
parameters.  Example:  The  linearized  equations  of  a crane  with 
physical  parameters  m^  ■ crab  mass,  m^  • load  mass,  £ * rope  length, 
g - gravitational  constant,  and  state  variables  x^  » crab  position, 
x^  “ crab  velocity,  x^  ■ rope  angle  and  x^  ■ rope  angular  velocity 


0 10  0 0 

0 0 m1g/mc  0 x + 1_  1 

0 0 0 1 “ “c  0 

0 0 -u)2  0 -1/ 


with  go  ■ (mc  + mt)g/mc£.  Input  u is  the  force  accelerating  the 
crab.  The  crane  may  operate  with  an  unknown  load  mass  m^  between 
the  empty  hook  and  the  maximum  mass,  for  which  the  crane  is  designed. 
It  may  also  operate  with  an  unknown  constant  rope  length  between  zero 
and  the  height  of  the  crane. 

6)  Numerically  known  dependence  of  a plant  model  on  uncertain  physical 
parameters.  Example:  linearized  equations  of  longitudinal  motion 
of  an  aircraft  depending  on  altitude  and  speed 


* - ^x  + BjU 


with  J typical  flight  conditions,  A ^ , B j , J ■ 1,2,...,J,  in  the 


flight  envelope. 
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Known  dynamics,  which  have  disappeared  in  a simplified  design  model 
by  linearization,  truncation  of  structural  modes,  model  reduction, 
neglecting  of  actuator  and  sensor  dynamics.  In  some  cases  it  may 
be  possible  to  pull  out  all  uncertainties  as  Illustrated  by  Fig.  1, 
where  for  P ■ 0 the  nominal  plant  N is  obtained. 


>1 

i ; ^ 

1 * 

N 

. 

v 

P 

L ' 

Fig.  1.  Nominal  plant  N with  perturbations  P. 

If  the  perturbations  P can  be  expressed  as  a 

diagonal  matrix  of  linear  or  nonlinear  operators, 

then  a robust  design  must  provide  sufficient  gain  and  phase  margins 

for  the  individual  loops  opened  at  v. 

8)  Unknown  dynamics,  which  cannot  be  modeled.  In  this  case  only  vague 
assumptions  about  perturbations  &A,  6B,  6C  of  the  system  matrices 

A,  B,  C (x  - Ax  + Bu,  y ■ Cx)  or  perturbations  6G(s)  of  the  transfer 
function  matrix  G(s)  can  be  made. 

9)  Quantization  effects  and  time  delays  in  controller  implementation. 

10)  Variance  of  components  in  mass  produced  control  systems  and  circuits. 
These  lists  of  system  properties  and  perturbations  show,  that  many  special 
combinations  can  be  specified.  Therefore  many  different  definitions  of 
"robust  control"  can  be  found  in  the  literature. 

The  design  problem  for  a robust  control  system  may  be  formulated  in  one 
of  the  following  three  forms: 
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1.  Given  a system  property.  Which  is  the  class  of  perturbations  with 
respect  to  which  the  system  property  is  robust?  Design  the  con- 
troller such  that  the  class  of  admissible  perturbations  is  extended 
in  the  direction  of  the  really  expected  perturbations. 

2.  Given  a class  of  perturbations.  Which  maximum  deviation  from  a 
desired  system  behavior  occurs  under  the  worst  perturbation  in  the 
given  class.  Design  the  controller  such  that  the  maximum  deviation 
is  minimized. 

3.  Given  a system  property  and  a class  of  perturbations.  Does  there 
exist  a set  of  controllers  for  which  the  system  property  is  robust 
under  the  class  of  perturbations?  If  yes,  select  one  on  other 
criteria  than  robustness.  If  no,  relax  specifications. 

2.  Controller  Structure 

Design  problems  for  control  systems  are  usually  parameterized  by  the 
assumption  of  a controller  structure  which  defines  a vector  of  design  para- 
meters. With  the  availability  of  cheap  computers  the  main  constraints  from 
the  side  of  controller  implementation  are  given  by  the  available  actuators 
and  sensors.  However  another  constraint  is  given  by  the  required  time  for  a 
particular  design.  This  of  course  depends  on  the  available  design  methods 
and  software  for  it. 

Two  typical  assumptions  for  the  controller  structure  are  adaptive 
controllers  or  fixed  gain  controllers  (e.g.  state  feedback,  dynamic  output 
feedback).  One  extreme  is  the  attempt  to  obtain  as  much  information  about 
the  perturbations  as  possible  by  on-line  identification  and  failure  detection. 
Then  ideally  the  structure  and  parameters  of  the  controller  are  adapted  in 
order  to  achieve  the  best  possible  performance  of  the  control  system  given  the 
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momentarily  available  Information.  An  intrinsic  difficulty  of  this  approach 
is  that  plant  inputs,  which  admit  a fast  and  accurate  identification,  are  not 
good  to  achieve  the  best  performance  and  vice  versa.  Also  a tradeoff 
between  a fast  failure  detection.  Identification  and  adaptation  and  a reliable 
one,  which  avoids  false  alarms  and  noise  sensitivity  of  the  adaptation,  must 
be  made. 

The  other  extreme  is  the  attempt  to  find  a fixed  gain  controller  which 
accomodates  a specified  class  of  perturbations.  In  this  approach  it  may  be 
necessary  to  sacrifice  some  performance  in  the  nominal  case  in  order  to 
achieve  robustness  for  the  perturbed  situation.  Only  this  case  is  usually 
called  robust  control,  however  it  should  be  apparent  from  the  previous  dis- 
cussion, that  robustness  is  a desirable  feature  also  for  an  adaptive  control 
system.  The  fixed  gain  solution  Indicates  whether  a more  complex  adaptive 
system  is  needed  at  all,  or  how  far  one  has  to  go  adaptive.  Practical 
solutions  to  the  robustness  problem  will  frequently  be  in  between  the  two 
extremes.  They  may  also  employ  variable  structures  with  state  dependent 
switching  between  fixed  linear  feedbacks  [30,31]. 

Frequency  domain  design  techniques  admit  any  dynamical  order  of  the 
controller.  State  space  design  methods  usually  assume  a state  feedback 
controller  structure.  For  a system  with  known  parameters,  all  Information 
which  is  relevant  for  its  future  dynamic  is  contained  in  the  present  state 
vector.  If  the  full  state  is  available,  then  the  processing  of  past  states, 
e.g.  in  dynamic  feedback  elements,  cannot  improve  the  performance  of  the  system. 
If,  however,  the  system  parameters  are  perturbed,  information  about  their 
actual  values  is  contained  in  past  states;  adaptive  systems  make  use  of  this 
fact.  In  this  situation  also  the  performance  of  fixed  gain  controllers  can 
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be  improved  by  processing  past  states  in  a dynamic  controller,  e.g.  in  dynamic 
output  feedback.  Uncertain  parameters  may  also  be  introduced  as  additional 
states,  which  may  be  estimated  and  fed  back. 

In  the  following  various  robustness  problems  will  be  discussed,  for 
which  at  least  partial  solutions  are  available  in  the  literature. 

3.  Sensitivity.  Robustness  With  Respect  to  Small  Parametric  Perturbations 

3.1.  Frequency  Domain  Methods 

The  main  reasons  for  the  use  of  feedback  are  stabilization  and  the  pre- 
servation of  desirable  system  properties  in  spite  of  noise  inputs  and 
perturbations  of  system  parameters. 

The  reduction  of  nonlinear  distortions  was  an  essential  reason  for  the 
use  of  feedback  amplifiers,  see  Black  [1].  The  reduction  of  nonlinearity  by 
high  gain  feedback  has  been  further  investigated  by  Cruz  [2]  and  Desoer  and 
Wang  [3]. 

In  frequency  design  methods  the  concept  to  compensate  the  loop,  such  that 
high  gains  are  possible  without  instability,  is  the  classic  rule  of  thumb  for 
the  reduction  of  noise  and  uncertainty.  Bode  [4]  expressed  it  in  terms  of 
gain  and  phase  margins  and  a sensitivity  function,  which  was  generalized  to 
the  multivariable  case  by  Cruz  and  Perkins  [3].  A sensitivity  matrix  S(s) 
relates  the  output  errors  Ec(s)  due  to  perturbations  in  a feedback  system  to 
the  output  errors  Eq(s)  due  to  the  same  perturbations  in  a corresponding  open 
loop  system  by  Ec(s)  » S(s)Eq(s).  The  sensitivity  matrix  S(s)  is  the  Inverse 
of  the  return  difference  matrix,  for  the  loop  of  Fig.  2. 

S(s)  - [I  + G(s)K(s)H(s)]‘1  (3) 
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Fig.  2.  Feedback  system,  return  difference  for  loops  broken  at  a. 

Note  that  G(s)  Is  the  actual  plant,  which  may  be  expressed  by  the  nominal 
design  model  G^(s)  and  a perturbation  60(3),  l.e.  G(s)  ■ GN(s)  + 6G(s).  If 
the  known  G^Cs)  Is  used  In  eq.  (3)  Instead  of  the  unknown  G(s),  then  all 
results  are  local,  l.e.  restricted  to  small  6G(a).  For  a reduction  of 
sensitivity  It  Is  sufficient  that 

S^(-ju))S(Jcu)  - I < 0 (neg.  semldeflnlte)  (4) 

over  the  frequency  band  of  Interest,  or  In  terms  of  the  return  difference 
F(s)  - I + G(s)K(s)H(s) 

FT(-)iu)F(ju>)  - I > 0 . (5) 

Hsu  and  Chen  [6]  proved  the  relationship 


det  F(s) 


closed  loop  characteristic  polynomial 
open  loop  characteristic  polynomial 


(6) 


Thus,  If  no  cancellations  occur,  closed  loop  stability  can  be  analyzed  using 
det  F(s).  MacFarlane  [7]  studied  the  eigenvalues  Pj(s),  j ■ l,2,...,m  of 
F(s)  and  showed  that  the  closed  loop  Is  stable,  If  all  characteristic  fre- 
quency loci  Pj(Jou),  j - l,2,...,m  satisfy  the  Nyqulst  criterion.  He  also 
proved  a necessary  condition  for  the  system  to  be  optimal  in  the  sense  of  a 
quadratic  criterion  ^ (yAQy  + u Ru)dt,  it  is 

|pj (Jiu)  | > 1 for  0 < oo  < • J • 1,2, . . . ,m  (7) 

or 
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|det  F(jtu)  | > l for  all  oj  . (8) 

These  results  have  the  graphical  interpretation  that  the  complex  plane  plots 
of  |det  F(joj)  | or  |pj(jcu)|  must  not  penetrate  the  interior  of  the  unit  disc. 

It  follows  from  this  that  the  characteristic  frequency  loci  of  an  optimal 
proportional  feedback  controller  have  infinite  gain  margin  and  at  least  60° 
phase  margin. 

Robustness  of  stability  with  respect  to  gain  and  phase  changes  may  also 
be  achieved  in  design  by  Rosenbrock's  inverse  Nyquist  array  [8].  Here 
I + Gq  *(ju))  with  Gq(s)  ■ G(s)K(s)H(s),  see  Fig.  2,  is  analyzed  graphically 
and  modified  in  the  design.  A standard  technique  in  multivariable  control 
system  design  is  to  use  compensation  or  feedback  to  decouple  or  approximately 
decouple  a multivariable  system  into  several  single  input  systems,  which  may  be 
designed  by  single-loop  techniques.  Rosenbrock  [8]  uses  the  criterion  of 
diagonal  dominance  for  approximate  decoupling. 

Doyle  showed  by  counterexamples  [9]  that  these  methods  can  lead  to 
highly  optimistic  margins  for  individual  loop  gains,  even  if  only  very  small 
margins  exist  for  simultaneous  change  of  several  loop  gains.  Already  in  the 
single-input  case,  gain  and  phase  margins  are  insufficient  to  characterize 
what  happens  for  simultaneous  gain  and  phase  perturbations.  Another  difficulty 
is  that  by  compensation  or  feedback  for  diagonal  dominance  the  actual  loca- 
tion of  the  uncertainty  is  obscured. 

Doyle  [9]  examines  the  properties  of  the  return  difference  using  the  con- 
cepts of  singular  values,  singular  vectors  and  the  spectral  norm  of  a matrix. 
The  singular  values  a ^ of  a matrix  A are  the  non-negative  square  roots  of  the 
eigenvalues  of  A*A,  where  A*  is  the  conjugate  transpose  of  A.  Since  A*A  is 
Hermitlan,  its  eigenvalues  are  real.  The  singular  values  give  a measure  of 
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how  close  A is  to  being  singular.  The  ratio  of  the  smallest  singular  value 
2 and  the  largest  one,  a,  is  the  condition  number  £/a«  One  may  also  inter- 
pret the  singular  values  as  generalizing  to  matrices  the  notion  of  gain. 

This  characterization  is  of  great  practical  value,  since  good  software  to 
compute  singular  values  is  widely  accessible  [10].  Using  this  singular  value 
concept  Doyle  proved  the  following  robustness  theorem: 


In  the  system  of  Fig.  3,  let  G(s)  be  rational,  square,  invertible  and  such 
that  the  nominal  closed  loop  with  L(s)  ■ 0 is  stable,  i.e.  G(I  + G)”^  • 

I + G 1 is  stable.  If  the  system  is  perturbed  by  L(s),  which  by  itself  is 
stable,  then  the  perturbed  system  is  stable  if 

^(I  + G L(ju>))  > a(L(ju)))  for  all  u>  . (9) 

For  this  theorem  Sandell  [11]  gave  a different  proof,  in  which  G(s)  need  not 
be  rational.  + G ^(ju>))  is  a frequency  dependent  measure  of  robustness 

in  terms  of  gain  margins.  For  the  eigenvalues  X of  A (here  ■ I + G_1(J(u)) 
generally  the  relation 

2(A)  < |X(A)|  < a(A)  (10) 

holds.  It  is  possible  that  the  smallest  eigenvalue  is  much  larger  than  2(A)* 
Thus  the  minimum  singular  value  2 gives  a more  reliable  measure  of  robustness 
than  the  smallest  eigenvalue.  In  fact  Doyle  constructed  an  example,  where 
the  diagonal  dominance  approach  as  well  as  the  characteristic  loci  approach 


generates  a Nyquist  or  Inverse  Nyquist  plot,  which  shows  + » db  gain  margin 
and  90°  phase  margin,  however  the  system  is  only  marginally  stable. 

On  the  other  hand,  singular  values  do  not  carry  phase  information,  they 
are  real,  no  Nyquist  type  encirclement  conditions  can  be  obtained. 

The  problem  of  uncertainties  due  to  a reduced  order  design  model  is 
interrelated  with  the  question,  which  modes  of  the  system  must  be  influenced 
by  the  control  and  which  others  snould  ideally  not  be  Influenced  at  all.  In 
vehicle  control  it  may  for  example  be  desirable  to  control  the  rigid  body 
dynamics  fast  and  accurately,  i.e.  with  a reasonably  high  bandwidth, without 
interferring  with  structural  vibrations.  In  frequency  domain  design  techni- 
ques, this  is  achieved  by  a 40  db/decade  roll  off  beyond  the  design  band- 
width. This  aspect  is  frequently  ignored  in  state  space  design  techniques. 
In  all  design  techniques  it  is  important  to  study  carefully  the  behavior  in 
a frequency  range  above  the  bandwidth,  where  modes  are  still  sufficiently 
controllable  and  observable,  such  that  the  control  may  move  them  into  the 


i 


; 


right  half  s plane. 

Stein  and  Doyle  [12]  give  a design  example  for  a CH-47  helicopter  with 
two  control  Inputs.  They  apply  singular  value  analysis  and  the  robustness 
condition  (9).  Rotor  dynamics  and  rate  limits  are  translated  into  a(L(ju>)) 
using  a result  of  Safonov  [13].  The  two  singular  values  were  made  approxi- 
mately equal  and  the  bandwidth  in  both  loops  was  increased  as  much  as  a(L(juu)) 
admitted.  A low  pass  helped  to  meet  the  "roll-off"  requirement.  The  example 
also  showed  that  these,  methods  may  lead  to  very  conservative  results  in  cases 
of  large  variations  of  parameters  in  specific  directions,  here  the  flight 
condition  variation. 
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3.2.  State  Space  Methods 

Single-input  linear  quadratic  state  feedback  regulators  have  a return 
difference  greater  than  unity  at  all  frequencies,  as  was  shown  by  Kalman  [14]. 
Anderson  and  Moore  [15]  showed  that  this  fact  implies  a + 60°  phase  margin, 
infinite  gain  margin  and  50  percent  gain  reduction  tolerance.  Safonov  and 
Athans  [16]  generalized  this  result  to  the  multiinput  case: 

x * Ax  + Bu 

(11) 

u * - Kx 


with  m inputs  u^  . 

The  feedback  matrix  K is  determined  by  solving  a Riccati  equation  minimizing 

00 

J - J (xTQx  + uTRu)dt  (12) 

0 

with  Q positive  definite  and  R * diag[r^, . . . .r^],  r^  > 0. 

The  individual  inputs  u^  are  perturbed  to  T^Uj^  without  interaction 
between  them,  i.e. 


x * Ax  + B7?u  with  7Ju  * ; 


Let  each  perturbation  7L  be  linear  time  invariant  with  proper  rational  stable 
transfer  function  P^s).  Its  frequency  response  is  P^ju))  * a^(u))*e 
Then  the  closed  loop  remains  stable  under  a phase  perturbation  with 

^((d)  | < 60°  for  all  id.  It  also  remains  stable  under  a gain  perturbation 
ai(u>)  >0*5  for  all  u>. 

Note  that  this  result  does  not  accomodate  neglected  actuator  dynamics  for 
two  reasons: 
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1.  Physical  actuators  have  at  least  90°  phase  lag  for  high  frequencies, 
this  can  only  theoretically  be  removed  by  feedback  of  actuator 
states,  which  in  turn  requires  modelling  of  the  actuator  as  part  of 
eq.  (1). 


2.  The  gain  and  phase  margins  do  not  apply  simulaitneously.  It  is 

known  from  the  single  input  case  that  gain  and  phase  margins  alone 
may  be  misleading.  The  two  Nyquist  curves  in  Fig.  4 both  have  60° 


phase  margin  and  infinite  gain  margin,  however  G^Cjuu)  has  a much 
smaller  distance  from  the  critical  point  >1  than  G^(jou). 


For  this  reason  Otto  Smith  [17]  used  the  "complex  gain  margin,"  i.e.  the  mini- 
mum distance  of  G(jou)  to  the  critical  point,  scaled  by  the  local  frequency  in- 
crement along  G(juu).  This  approximates  the  negative  real  part  of  a dominant 
pair  of  eigenvalues.  A multivariable  measure  for  the  distance  of  G(Jcu)  from 
the  critical  point  has  been  discussed  already  in  form  of  the  singular  values 
of  the  return  difference. 

Doyle  [18]  showed  by  counterexample  that  the  margins  may  be  arbitrary 
small,  if  the  state  is  replaced  by  a state  estimate  from  a Kalman  filter.  In 
his  example,  the  gain  margins  were  arbitrarily  small  in  both  the  positive  and 
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negative  db  direction.  To  improve  the  margin  in  this  situation,  Doyle  and 
Stein  [19]  developed  a "design  adjustment  procedure,"  which  introduces 
fictituous  noise  at  the  control  input  to  the  plant.  In  this  procedure  the 
observer  eigenvalues  tend  to  the  finite  transmission  zeros  and  to  infinity. 

Thus  the  procedure  works  only  for  minimum-phase  plants.  The  procedure  is 
essentially  the  dual  of  Kwakernaak's  sensitivity  recovery  method  [20].  This 
however  drives  the  plant  poles  instead  of  the  observer  poles  to  the  trans- 
mission zeros,  which  may  lead  to  large  control  inputs  u. 

The  solution  of  the  Riccati  equation  allows  a pole  far  left  in  the  s- 
plane.  Also  in  pole  placement  techniques  such  a pole  assures  that  the  system 
is  optimal  [15].  This  pole  is  not  desirable  if  the  model  uncertainties  increase 
with  frequency  and  a "roll-off"  bandwidth  limitation  is  needed.  In  this 
situation  all  controlled  eigenvalues  should  be  kept  Inside  and  on  a semi- 
circle in  the  left  half  s plane,  the  radius  of  which  is  the  design  bandwidth. 

Gain  and  phase  margins  may  be  much  smaller  in  discrete  time  linear 
quadratic  state  feedback  systems.  Jacques  Willems  and  van  de  Voorde  [21] 
give  bounds  for  the  single-input  case,  which  show  that  the  system  may  be  very 
sensitive  to  feedback  gain  variations.  This  is  not  surprising,  since  the  hold 
element  may  be  approximated  by  a phase  shift  of  one  half  sampling  interval. 

Safonov  and  Athans  [16]  also  generalize  a single- input  result  by  Anderson 
and  Moore  [15],  which  is  useful  for  actuator  nonlinearities.  If  the  perturba- 
tion operator  71  in  eq.  (13)  describes  a time  varying,  memory  less  nonlinearity 
7?iui  * fi(u,t),  then  it  is  a sufficient  condition  for  the  closed  loop 
stability,  that 

\ ~ f(u,t)  < M 


for  some  M < « and  for  all  t . 


(14) 
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For  example  for  an  actuator  saturation,  stability  is  guaranteed  if  the  inputs 
do  not  exceed  twice  the  saturation  level. 

Comparisons  of  numerous  optimization  techniques  for  insensitive  control 
systems  were  made  by  Harvey  and  Pope  [22,23]  for  wing  load  alleviation  for  the 
C-5A  aircraft  and  by  Vinkler  and  Wood  [24]  for  a lateral  autopilot  for  a 
rudderless  remotely  piloted  vehicle.  A minimax  technique  by  Salmon  [25]  and 
an  uncertainty  weighting  technique  by  Porter  [22]  were  judged  superior  to  six 

other  techniques  in  the  first  report,  both  however  failed  in  the  comparison 

« 

[27].  Here  an  expected  cost  technique  by  Ly  and  Cannon  [26]  and  a multistep 
guaranteed  cost  technique  by  Vinkler  and  Wood  [27]  came  out  better  than  four 
other  techniques.  In  [23]  an  information  matrix  approach  by  Kleinmann  and 
Rao  [28]  compared  favorably  with  other  techniques. 

In  problems  with  insignificant  constraints  on  the  control  inputs,  the 
weighting  matrix  R in  a quadratic  criterion  may  be  small.  This  leads  to  high 
gain  solutions  as  they  were  discussed  in  the  previous  section.  A comparison 
of  various  high  gain  feedback  systems  is  made  by  Young,  Kokotovic  and  Utkin 
[29].  This  comparison  also  includes  variable  structure  systems,  which  in 
their  sliding  mode  are  insensitive  to  parameter  variations  and  disturbances, 
similar  to  the  high-gain  system  [30].  Young  [31]  applied  this  concept  to  the 
design  of  an  adaptive  model  following  control  system  and  compared  the  results 
for  the  longitudinal  motion  of  a Convair  C-131B  aircraft  with  other  model 
following  techniques. 

A special  case  of  a high  gain  control  system  is  useful,  if  the 
reference  or  disturbance  input  signals  can  be  exactly  modelled  and  asymptotic 
tracking  or  disturbance  rejection  is  required.  The  use  of  integrators 
in  the  loop  for  zero  stationary  errors  in  step  and  ramp  responses  is  a 
classical  recipe.  Also  for  other  Inputs  an  internal  model  of  the  input 


can  be  used,  e.g.  a tuned  oscillator  (notch  filter)  for  disturbance  rejection 
of  helicopter  rotor  vibrations,  whose  frequency  is  regulated.  Such  a high 
gain  at  particular  frequencies  makes  asymptotic  tracking  robust  to  plant  para- 
meter variations  as  long  as  the  loop  remains  stable.  This  robustness  problem 
was  studied  by  Davison  [32]  and  others.  In  sampled-data  systems  the  Internal 
model  is  to  be  implemented  in  continuous  time,  if  the  tracking  property  is 
required  also  between  the  sampling  instants  [33]. 

Some  common  problems  in  all  high  gain  concepts  are 

* Measurement  noise  goes  highly  amplified  to  the  actuator  inputs. 

* High  values  for  |u | and  |u|  may  occur. 

* Non-cooperative  efforts  of  the  actuators  may  occur. 

The  LQG  design  method  offers  a systematic  way  to  avoid  these  difficulties  by 
increase  in  the  R matrix  and  by  the  use  of  a Kalman  filter. 

4.  Robustness  With  Respect  to  Large  Perturbations  in  Known  Directions. 

In  the  methods  of  Section  3 relatively  little  knowledge  about  the  para- 
metric perturbation  is  assumed.  The  results  are  therefore  primarily  valid  for 
small  perturbations.  In  some  cases  information  is  obtained,  how  big  the 
perturbation  is  allowed  to  be  in  order  to  maintain  stability. 

In  situations  where  large  perturbations  in  known  directions  occur,  the 
previous  methods  generally  lead  to  very  conservative  results.  In  this  section 
some  tools  are  discussed  by  which  such  perturbations  can  be  accomodated  in 
the  design. 

In  [34]  parameter  space  methods  are  used  for  design.  Single-input  pole 
placement  is  formulated  as  a linear  map  from  the  parameter  space  g of  co- 
efficients of  the  characteristic  polynomial  into  the  parameter  space  X of 
state  feedback  gains.  It  is  shown  that  a characteristic  polynomial 


det (\I  - A + bk' ) - p + p \ + 


t Tollable  pair  A.b  by 


where 


and  e'  Is  Che  last  row  of  Che  InverCed  cocrollabilicy  macrtx  [33]. 

Example  1:  Gain  scheduling  for  che  crane  of  eq.  (1)  for  variable  load 


A a (p. l/g  - p.).  The  characCerlsCic  polynomial  remains 


invarlanC  under  large  load  changes  if  k , k and  k,  are  conscanc  as  given  and 


k^  is  scheduled  by  Che  load  according  Co  k^  ■ k^Q  + tn^g. 

Example  2:  Small  gain  scabilizaclon  of  all  cranes  by  ouCpuC  feedback 


The  open  loop  characCerlsCic  polynomial  deC(sI-A)  - s (s  -Ku  ) wlch  four 


eigenvalues  on  Che  imaginary  axis  is  scabilized  Co  P(s)  * deC(sI-A+bk' ) 


(s  +as+b)(s  +CS+U)  +d)  wich  small  a>0,  b>0,  c > 0 & small  d by  k 


£(m  b-m  d),  k,  ■ £(m, a-m  c).  This  describes  Che  cone  of 


scabilizlng  direcCions  aC  Che  origin  of  che  four  dimensional  3C  space.  IC  in' 


eludes  che  direcCions  k,  • 0 and  k,  > 0,  i.e.  no  feedback  of  che  rope  angle 


and  rope  angular  velocicy  is  necessary  if  d - bm./m  and  c • am  /m  are 


chosen.  Thus  k'  ■ [k.  k.  0 0]  wlch  small  k.  > 0 and  k.  > 0 assigns  Che 


characCerlsCic  polynomial  P(s)  ■ (s  +as+b)(s  +(mi/mc)as  + w + bmi/mc),  where 

a - (mc+m^)  and  b » ki/  (mc+m^)  • The  resulc  is  Chac  outpuC  feedback 

k'  - [k.  k.  0 0]  scabilizes  all  cranes  wich  arbicrary  posidve  physical  para- 
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In  the  second  example  of  a globally  robust  system  the  eigenvalues  are 
moved  only  Incrementally  from  their  widely  varying  open  loop  position.  This 
is  in  agreement  with  a rule  of  thumb:  If  you  have  to  care  about  constraints 
on  |u  | or  |u|,  do  not  try  to  make  a slow  system  fast  or  a fast  system  slow. 

In  other  words,  under  large  parameter  variations  it  is  not  desirable  to  have 
only  one  desired  reference  model  or  reference  trajectory.  Also  practically 
no  pilot  would  expect  that  an  aircraft  has  the  same  dynamics  in  all  flight 
conditions.  For  the  crane  it  is  not  necessary  to  have  the  same  eigenvalues 
under  all  loads  like  in  the  gain  scheduling  system  of  Example  1,  it  is 
sufficient  to  have  a minimum  damping  and  minimum  negative  real  part  of  the 
eigenvalues  for  the  load  range  from  the  empty  hook  to  the  maximum  load,  for 
which  the  crane  is  built.  This  suggests  the  idea  to  specify  a region  in  the 
eigenvalue  plane,  in  which  the  eigenvalues  shall  remain  under  large  parameter 
variations,  instead  of  a fixed  set  of  eigenvalues.  In  [34]  the  boundaries  of 
such  regions  are  mapped  into  the  gain  space  3C.  In  JC  the  region  is  determined 
in  which  the  feedback  gains  must  be  chosen,  such  that  all  eigenvalues  are  in 
the  specified  region  in  the  eigenvalue  plane.  If  J pairs  A^.bj  are  given 
(see  eq.  (2)),  then  for  each  pair  a region  in  X space  is  obtained.  A fixed 
gain  solution  does  exist  if  all  J regions  have  a cotnnon  intersection.  This 
intersection  gives  an  admissible  set  of  feedbacks.  A particular  element  can 
then  be  selected  under  other  aspects,  e.g.  such  as  to  minimize  the  norm 
/k' k,  i.e.  the  distance  from  the  origin  in  X space. 

Example  3 [35]:  The  F4-E  aircraft  with  horizontal  canards  has  a flight 
envelope  as  shown  in  Fig.  5.  Four  typical  flight  conditions  were  chosen  for 
a study.  Figure  6 shows  the  open  loop  eigenvalues  of  the  longitudlal  short 
period  mode. 
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The  aircraft  is  unstable  in  the  subsonic  flight  conditions  1,  2,  3 and 
insufficiently  damped  in  the  supersonic  flight  condition  4.  Thus  feedback 
must  improve  the  handling  qualities  in  all  flight  conditions  in  order  to  meet 
the  military  specifications  for  the  short  period  eigenvalue  location.  These 
are  shown  in  Fig.  6 for  flight  condition  ©•  The  frequency  ranges  vary  with 
the  flight  condition.  For  flight  condition  1 this  region  in  the  eigenvalue 
plane  maps  into  the  region  Q in  the  plane  of  accelerometer  and  gyro  feedback 
gains  in  Fig.  7.  Doing  the  same  for  flight  conditions  2,  3 and  4 and  taking 
the  intersection  gives  the  shaded  region  in  Fig.  7.  For  all  pairs  of  gains 
inside  the  shaded  region  the  military  specifications  are  satisfied  for  all 
flight  conditions. 

Fam  and  Meditch  [36]  showed  a useful  result  for  stability  of  discrete 
time  systems:  The  convex  hull  of  the  stability  region  in  the  parameter  space 
9 of  characteristic  polynomial  coefficients  is  a polyhedron,  whose  vertices 
correspond  to  the  n+1  polynomials  P(z)  with  zeros  in  the  set  [-1,1}.  In  [34] 
this  polyhedron  is  mapped  into  the  X space.  Due  to  the  linearity  of  k1  ■ p'E 

the  region  in  X space  is  also  a polyhedron  and  its  n+1  vertices  are  obtained 

t-r 

by  pole  placement  Jm  the  vertices  in  P space.  For  each  pair  A^.b^, 
j ■ 1,...,J,  a different  polyhedron  is  obtained.  A necessary  condition  for 
k'  to  stabilize  simultaneously 

Pj(z)  - det(zl  - Aj  + bjk' ) j - 1 J (16) 

is  that  k'  is  in  the  intersection  of  the  J polyhedra  in  X space,  which  is  it- 
self a polyhedron.  If  the  regions  do  not  have  a common  Intersection,  then 
there  does  not  exist  such  k' . 

The  sufficient  conditions  are  much  more  difficult.  The  set  of  admissible 
solutions  to  (16),  if  it  exists,  is  generally  not  convex  and  not  even 
connected,  such  that  a search  inside  the  Intersecting  polyhedron  must  be  made. 
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Admissible  regions  of  feedback  gains 
satisfying  the  military  specifications 
for  short  period  eigenvalue  locations 
in  four  flight  conditions.  S 
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In  numerical  techniques  it  is  convenient  to  formulate  an  optimization 
problem  in  such  a way  that  there  always  exists  a solution,  which  is  then 
iteratively  improved.  The  problem  of  disconnected  sets  then  becomes  one  of 
local  minima. 

In  typical  design  examples  not  only  the  mathematical  model  of  the  plant 
is  uncertain,  but  also  the  formulation  and  relative  weight  of  many  design 
criteria.  Some  of  these  criteria  are  in  form  of  inequality  constraints  others 
are  to  be  minimized.  It  is  very  artificial  to  put  all  of  them  together  into 
one  scalar  performance  index,  which  is  then  minimized  over  the  parameters  in 
an  assumed  controller  structure.  For  the  designer  an  interactive  computer- 
aided  design  procedure  is  more  useful,  where  he  can  make  higher  level  decisions 
of  how  to  change  requirements  after  each  computer  solution  or  failure  to  find 
a solution.  The  computer  may  have  to  solve  a nonlinear  programming  problem  in 
each  design  step.  Various  aerospace  problems  have  been  formulated  and  solved 
this  way.  Schy  [37,38]  deals  with  a lateral  stability  augmentation  system  for 
a fighter  airplane,  Hauser  [39]  with  an  autopilot  for  a flexible  space  vehicle. 
Further  design  examples  are  given  by  Karmarkar  [40]  and  Kanarachos  [41].  It 
is  convenient  to  formulate  all  design  criteria  for  each  operating  point  as 
components  of  a performance  vector  &.  It  may,  for  example,  contain 

• bounds  on  the  individual  feedback  gains  |k^  | 
and  for  each  flight  condition  specifications  on 

• eigenvalue  location. 

* deviation  from  nominal  response  for  typical  reference  and  disturbance 
Inputs. 

* bounds  on  the  control  rate  |u|  for  typical  reference  and  disturbance 
Inputs . 


Kreisselmeier  and  Steinhauser  [42]  use  in  an  example  with  five  flight  conditions 
of  a F4-C  aircraft  a 40  dimensional  vector  £.  A vector  constraint  £ 

(i.e.  componentwise  g^  < c^  is  given  and  the  feedback  gains  K are  the 


solution  of  the  problem 


Min [Max  g. (K)/c.}  . 
K i 1 1 


I 


Using  an  algorithm  described  in  [43]  Kreisselmeier  and  Steinhauser  obtain  a 
Pareto-optimal  solution.  Figure  8 shows  some  reference  step  responses  of 
this  design  for  an  F4-C.  It  is  stable  in  the  five  flight  conditions.  The 
open  loop  responses  on  the  left  side  show  that  the  aircraft  is  slow  in  flight 
condition  1 (landing  approach).  Here  a slower  reference  response  was  given 
than  for  the  high  speed  conditions  2 and  4.  The  desired  reference  response 
was  specified  as  g^(t)  ■ g^(a^t)  where  for  each  flight  condition  i * 1,2,... ,5 
an  appropriate  time  scale  was  chosen.  This  resulted  in  the  insensitive 
closed  loop  responses  on  the  right  side  of  Fig.  8,  which  required  only  a 
relatively  small  control  rate  |u|.  The  same  feedback  resulted  in  similarly 
good  disturbance  responses. 

Also  the  results  of  Shy  [38]  showed  that  an  amazingly  large  variation  of 
parameters  can  be  accomodated  by  a fixed  gain  controller,  if  the  requirements 
were  in  good  agreement  with  the  physical  limitations.  These  designs  result 
in  low  gain  solutions,  and  the  dynamics  change  in  an  acceptable  or  desirable 
way  as  the  physical  parameters  vary. 

5.  Integrity.  Robustness  With  Respect  to  Sensor  and  Actuator  Failures 


If  an  actuator  or  sensor  is  connected  to  a high  gain,  then  its  failure 
is  a larger  perturbation  than  in  a low  gain  situation.  Thus  requirements  for 
robustness  with  respect  to  actuator  and  sensor  failures  tend  to  result  in  low 
gain  solutions.  Even  more  important  is  the  aspect  of  avoiding  non-cooperative 
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efforts  of  actuators.  If,  for  example,  one  input  alone  places  some  eigen- 
values in  the  right  half  plane  and  another  one  is  needed  to  bring  them  back 
into  the  left  half  plane,  then  apparently  no  robustness  of  stability  with 
respect  to  actuator  failures  can  be  achieved. 

One  approach  to  achieve  robustness  of  stability  with  respect  to  certain 
failures  is  to  try  to  extend  gain  reduction  margins  to  include  gain  zero. 
Belletrutti  and  MacFarlane  [44]  use  the  term  "high  integrity"  for  robustness 
with  respect  to  certain  failures.  They  check  the  stability  conditions  for 
gains  reduced  to  a small  « using  Nyquist  stability  criteria  for  character- 
istic loci  of  principal  submatrices  of  the  return  ratio.  In  this  analysis 
the  loop  must  be  broken  at  the  point  where  the  actual  failure  may  occur  and 
thus  the  gain  reduction  margin  is  needed.  Owens  [45]  derived  necessary  and 
sufficient  conditions  for  integrity  of  systems  with  multivariable  proportional- 
integral  controllers.  r 

Solheim  [46]  formulated  the  integrity  problem  in  the  context  of  quad- 
ratic optimal  control.  In  examples  an  increased  integrity  is  obtained  with 
an  increased  weight  R on  the  control  in  the  quadratic  criterion,  another 
indication  that  the  solution  will  tend  to  a low  gain  solution.  Wong,  Stein 
and  Athens  [47]  show  the  following  gain  reduction  result  for  LQ  regulators: 

The  matrix  A£(A)  » A + BAR  with  A ■ diag[a^. . .0^],  where  K minimizes 

ao 

J x'Qx  + u'Rxdt  for  A ■ I,  is  stable  for  all 

A>-|  [I  - (R1/2  K'Q'W72)*1]  . (18) 


This  generalizes  the  bound  > 0.5  from  [16].  The  recommendation  is,  from  a 
purely  robustness  standpoint,  to  choose  Q and  R such  as  to  maximize 


min 


{(R1/2K'Q* lKR1/2)*1)  . 


(19) 


I 

j 

I 
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Kreisselmeier  [48]  proposes  to  modify  the  quadratic  criterion,  where  for  each 
considered  failure  situation,  a quadratic  criterion  Is  formulated  and  the 
overall  criterion  is  a weighted  sum  of  these  terms. 

In  failure  situations  it  may  be  desirable  to  specify  other  emergency 
boundaries  in  the  eigenvalue  plane  than  only  the  Imaginary  axis.  This  problem 
is  treated  by  parameter  space  methods  in  [34].  The  concept  is  illustrated 
for  the  case  of  sensor  failures  in  Fig.  9.  A nominal  region  for  the  eigen- 
value location  and  a larger  emergency  region  are  mapped  into  the  space  of 


Fig.  9.  Illustration  of  failure  robustness  and 
emergency  boundaries. 

feedback  gains.  It  is  assumed  that  the  system  is  represented  in  "sensor 
coordinates,"  then  a failure  of  a sensor  for  state  variable  x^  corresponds  to 
switching  k^  to  zero.  The  projection  of  point  1 on  the  k^  axis  is  outside 
the  emergency  boundary,  i.e.  the  emergency  specification  is  not  robust  with 
respect  to  a sensor  failure  k£  * 0.  It  is,  however,  robust  with  respect  to 
k^  ■ 0.  For  all  points  in  the  shaded  area  the  emergency  specifications  are 
robust  with  respect  to  either  sensor  failure.  An  alternative  to  this  robust 
solution  would  be  in  this  example  to  omit  sensor^  1 and  to  use  multiplexed 


i 


sensors  for  and  failure  detection. 

In  the  multiinput  case  a sensor  failure  is  equivalent  to  changing  a 
column  of  the  K matrix  to  zero  and  an  actuator  failure  is  equivalent  to 
changing  a row  of  K to  zero.  In  [34]  an  actuator  failure  example  is  studied, 
where  the  problem  is  formulated  such  that  the  eigenvalues  are  placed  in  a 
nominal  position  with  two  actuators  and  move  as  little  as  possible  towards 
the  stability  boundary  for  failures  of  either  one  of  two  actuators. 

Apparently  a necessary  condition  for  robustness  with  respect  to  failures 
is  that  the  insufficiently  damped  eigenvalues  (outside  the  specified  region) 
remain  controllable  and  observable  after  the  failure.  In  the  crane  example, 
the  sensor  for  the  crab  position  x^  is  essential,  because  x^  is  not  observable 
by  other  states.  In  such  situations  it  is  apparently  misleading  to  use  high 
gain  feedback  and  to  show  gain  reduction  to  only  a few  percent  of  the  high 
gain.  For  failures  of  essential  actuators  and  sensors  only  redundant 


components  can  help. 
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Aba tract 

Model  Reference  Adaptive  Systems  (MRAC)  where  the  control  objectives  are 
specified  by  a reference  model  and  Stochastic  Self-Tuning  Regulators  (S-STURE) 
where  the  control  objectives  are  specified  by  an  A.R.M.A.  model  are  presented 
together  from  a duality  point  of  view.  The  duality  existing  between  these  two 
classes  of  adaptive  systems  extends  the  duality  existing  in  the  linear  case 
with  known  parameters  between  minimum  variance  control  and  modal  control. 

The  equivalence  between  Implicit  and  Explicit  M.R.A.C.  is  discussed  and 
the  corresponding  S-STURE  with  explicit  and  Implicit  prediction  reference 
models  are  defined.  Tools  for  analysis  of  M.R.A.C.  and  S-SiJRE  in  a deter- 
ministic and  a stochastic  environment  respectively  are  given.  They  are  also 
used  together  with  the  duality  aspects  to  explore  the  behaviour  of  M.R.A.C.  in 
a stochastic  environment  and  the  behaviour  of  S-STURE  in  a deterministic 
environment. 

Finally,  the  design  of  schemes  behaving  as  a desired  M.R.A.C.  in  a deter- 
ministic environment  and  as  a desired  S-STURE  in  a stochastic  environment  is 
indicated  and  an  example  is  given. 

1.  INTRODUCTION 

Recent  works  [1],  [2],  [3]  have  enlighted  the  connections  existing  between 
adaptive  control  systems  with  an  explicit  reference  model  (called  also  direct 
adaptive  control  [2])  where  the  parameters  of  the  controller  are  directly 
adapted  and  the  adaptive  control  systems  with  an  implicit  reference  model 
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(called  also  indirect  adaptive  control  [2])  where  an  adaptive  predictor  derived 
from  M.R.A.S.  techniques  is  used  as  an  intermediate  step  and  the  parameters 
of  the  adaptive  predictor  are  used  to  up-date  the  controller.  These  two  schemes 
designed  from  a stability  point  of  view,  to  operate  in  a deterministic  environ- 
ment can  be  equivalent.  The  first  condition  which  should  be  satisfied  in  order 
that  the  two  schemes  be  equivalent  is  that  the  control  strategy  for  the  implicit 
M.R.A.C.  is  such  that  the  output  of  the  adaptive  predictor  behaves  identically 
to  that  of  the  explicit  reference  model  (i.e.  the  adaptive  predictor  plus  the 
controller  form  and  implicit  reference  model). 

In  refs,  [l],  [4],  connections  between  M.R.A.C.  designed  to  operate  in  a 
deterministic  environment  and  the  Minimum  Variance-Self  Tuning  Regulator 
(MV-STURE)  designed  to  operate  in  a stochastic  environment  have  been  investi- 
gated (the  MV-STURE  has  a structure  similar  to  the  implicit  M.R.A.C.).  Further- 
more as  it  will  be  shown  in  this  papeq  minimum  variance  control  and  respectively 
MV-STURE  appear  as  particular  cases  of  stochastic  control  and  stochastic  STURE 
where  the  control  objectives  are  specified  by  an  ARMA  model. 

Therefore,  the  basis  for  a unified  approach  to  M.R.A.C.  and  S -STURE  exist. 

Work  in  this  direction  have  been  done  in  [S]  where  the  on-line  parameter 
estimation  via  prediction  error  methods  have  been  emphasized  as  the  common 
interpretation  of  the  various  schemes.  Another  work  towards  a unified  approach 
to  M.R.A.C.  and  S-STURE  on  which  the  present  paper  is  largely  based  is  presented 
in  [6],  where  the  duality  between  linear  deterministic  modal  control  and  minimum 
variance  stochastic  control  has  been  extended  to  M.R.A.C.  and  S-STURE  and  the 
interpretation  of  S-STURE  as  "stochastic"  M.R.A.C.  has  been  sketched. 

In  the  present  paper,  the  ideas  from  [6]  are  developed;  structural 

n 

similarities  duality  and  equivalence  as  well  as  the  differences  between  various 
schemes  are  more  deeply  investigated.  This  allows: 
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1)  To  analyse  In  a straightforward  way  the  behaviour  of  M.R.A.C.  in  a 
stochastic  environment  and  vice-versa  the  behaviour  of  S-STURE  in  a deter- 
ministic environment. 

2)  To  define  new  schemes  of  M.R.A.C.  and  S-STURE  which  can  offer  better 
performances  in  some  situations. 

3)  To  design  adaptive  control  schemes  which  can  operate  in  a deterministic 
environment  as  a desired  M.R.A.C.  and  in  a stochastic  environment  as  a S-STURE. 

The  common  denominators  for  this  unified  approach  for  M.R.A.C.  and  S-STURE 
are  the  structure  of  the  parameter  adaptation  algorithm  and  the  presence  of  a 
reference  model  (implicit  or  explicit)  in  the  deterministic  environment  and  its 
counterpart  the  prediction  reference  model  (implicit  or  explicit)  in  the 
stochastic  environment. 

The  paper  is  organized  as  follows.  Section  2 recalls  the  duality  between 
minimum  variance  control  and  stochastic  control.  In  Section  3,  this  duality 
is  extended  for  the  case  of  Linear  Model  Following  Control  where  the  design 
objectives  are  specified  by  a difference  equation  and  a class  of  linear 
stochastic  controllers  where  the  design  objectives  are  specified  by  an  ARMA 
model.  In  Section  4,  the  principles  and  block  diagrams  of  Explicit  M.R.A.C., 
Implicit  M.R.A.C.  and  S-STURE,  which  are  extensions  of  the  linear  control 
strategies  discussed  in  Section  3 when  the  plant  parameters  are  unknown,  are 
reviewed  and  the  structural  similarities  are  emphasized.  In  Section  5,  tools 
for  analysis  of  the  various  schemes  in  a deterministic  environment  and  a 
stochastic  environment  are  given.  These  tools  are  based  on  the  EFR  method 
(Equivalent  Feedback  Representation)  and  the  ODE  method  (Ordinary  Differential 
Equation)  respectively.  In  Section  6,  a MV-STURE  scheme  and  an  Implicit 
M.R.A.C.  scheme  are  presented  and  they  will  serve  as  a basis  for  illustrating 
the  properties  of  various  configurations.  In  Section  7,  the  equivalence  between 
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implicit  and  explicit  M.R.A.C.  is  discussed.  In  Section  8,  the  asymptotic 
duality  between  M.R.A.C.  and  S-STURE  is  examined  (and  M.R.A.C.  dual  to  MV-STURE 
are  constructed).  In  Section  9,  the  interpretation  of  S-STURE  as  stochastic 
M.R.A.C.  using  ai  implicit  reference  prediction  model  and  a new  equivalent 
realization  of  S-STURE  using  an  explicit  reference  prediction  model  are  given. 

In  Section  10,  the  problem  of  the  positivity  conditions  required  for  the  con- 
vergence of  both  M.R.A.C.  and  S-STURE  is  examined  in  connection  with  a converse 
dual  problem.  In  Sections  11  and  12,  the  behaviour  of  MRAC  in  a stochastic 
environment  and  of  S-STURE  in  a deterministic  environment  is  examined  in 
connection  with  the  duality  MRAC-S-STURE.  A combined  MRAC-S-STURE  scheme  is 
then  derived  in  Section  13.  This  scheme  can  operate  in  a deterministic  and 
stochastic  environment  as  a MRAC  and  a S-STURE  respectively. 

In  order  to  reduce  the  technical  details  and  to  make  the  presentation  more 
transparent,  throughout  the  paper  the  plant  to  be  controlled  in  the  absence  of 
disturbances  is  assumed  to  be  described  by  a discrete  rational  transfer  function 
with  a basic  delay  of  one  sample  and  with  the  leading  coefficient  of  the 
numerator  being  known  and  constant.  All  the  results  can  be  extended  for  the 
case  of  a general  delay  and  unknown  numerator  leading  coefficient  and  this  will 
be  the  object  of  a forthcoming  paper. 


2.  THE  DUALITY  BETWEEN  MINIMUM  VARIANCE  CONTROL  AND  THE  MODAL  CONTROL 
2.1  Stochastic  case 

Consider  the  process  to  be  controlled  and  its  stochastic  environment 
described  by: 


y . u + C-Ia'-h  v .ray  + ? b u 

yk  -1.  k-1  -1.  k aiyk-l  + \ i k-i-1 

A(q  ) A(q  ) i-1  i-1 


+ b u 


-lx 


oVl  - £ Vk-l  + vk  • Vk-1  + bo“k-l  + *<«  >vk  (2-L> 
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where  y^  is  Che  measured  output,  is  Che  conCrol  and  v^  is  a sequence  of 
equally  disCribuCed  independent  normal  (0,  a)  random  variables  and: 


ACq"1)  - 1 - aiq-1  ...  - anq_CI  (2.2) 

B(q_1)  - bQ  + b^"1  - ...  + bmq’m  (2.3) 

C (q”1)  “ 1 - c^'1  ...  - cnq‘n  (2.4) 

P0  " £*1  *n*  bl  bm^  (2,5) 

«Ll  " ^k-1  Vn'  uk-2  Vm-ll  (2‘6> 


The  polynomial  B(z**)  and  C(z~*)  are  supposed  Co  have  all  zeros  in 


The  minimum  variance  conCrol  is  calculaCed  such  ChaC  Che  following 

2 

objecCives  are  meC:  1)  E{y^}  * 0,  2)  Ety^}  * min.  The  minimum  variance 
conCrol  can  be  calculaCed  direccly  [7]  and  one  geCs: 


where: 


Vl  “ ■ b tPMV  ^.1.] 

o 


* a. "C.  ...  a * c | b,  ...  b 
MV  11  n n*  1 n 


(2.7) 


(2.8) 


2 2 

WiCh  chis  conCrol,  E[y.  } » E{vj"}  and  lim  y.  ■ v.  . 


'JqJ  — v j muw  a mm  j 

k -•  oo 

For  Che  beCCer  understanding  of  Che  minimum  variance  self-tuning  regulator 
and  iCs  connections  with  M.R.A.S.,  it  is  worth  Co  recall  that  Che  minimum 
variance  control  can  be  obtained  using  Che  separation  theorem,  l.e.: 

1)  Design  an  optimal  predictor  from  (2.1). 


^k/k-1  “ PO0k-l  + bOuk-l  ‘ iJ1  civk-i 


(2.9) 


2)  Determine  a control  for  the  predictor  (2.7)  in  order  to  achieve  Che 
deterministic  objective  a 0 (and  take  advantage  that  in  this  case  y^  » v^) 
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This  strategy  to  compute  the  MV  control  will  then  be  extended  In  order  to 
obtain  a MV-STURE  when  the  plant  parameters  and  disturbance  parameters  are 


unknown. 


Note  also  that  the  minimum  variance  control  strategy  can  be  formulated 
In  a different  way  which  will  be  exploited  later:  for  the  process  and  the 
stochastic  disturbance  given  in  (2.1),  find  u^  such  that  lim  yfc  ■ v^. 

k -•  SB 


2.2  Deterministic  case 


The  process  to  be  controlled  is  described  by: 


\ • Vk-i  + boVi  ; y(0)  * 0 


(2.10) 


where  Pq  and  0^^  are  8iven  hy  (2.5)  and  (2.6). 

The  objective  of  the  "modal  control"  is  either  to  find  u^  such  that 
yk  s 0,  V k > 1 (all  the  closed  loop  poles  are  at  the  origin)  or  such  that: 


where: 


A°(q"1)yk  - 0 


.0,  -1.  .0-1  ,0  -n 

A (q  ) - 1 - axq  ...  - aQq 


(2.11) 


(2.12) 


defines  the  desired  poles  of  the  closed  loop  system.  A simple  analysis  shows 


that  the  modal  control  is: 


Uk-1  " " bQ  ^McA-l^ 


where 


PMC  " CVal'  **•  VV  bl  •••  bn]  * 


(2.13) 


(2.14) 


If  now  A^(q”*)  • C(q”*)  where  C(q~*)  defines  the  stochastic  disturbance 
in  (2.1),  the  controls  given  by  (2.7)  and  (2.13)  are  the  same.  One  has  there- 


fore the  following  result. 


THEOREM  2.1:  (Duality  between  minimum  variance  control  and  modal  control) 


The  minimum  variance  control  of  the  process  and  its  stochastic  environment 


(2.1)  is  identical  to  the  modal  control  for  the  same  process  in  a deterministic 


environment  if  and  only  if  the  desired  closed  loop  behaviour  is  defined  by 


3.  LINEAR  MODEL  FOLLOWING  CONTROL  AND  LINEAR  STOCHASTIC  (ARMA)  MODEL 
FOLLOWING  CONTROL 


The  remarks  made  in  Section  2 can  be  generalized  as  it  will  be  shown  next 


3.1  Deterministic  case 


In  the  case  of  a deterministic  environment  not  only  the  regulation 


objectives  are  specified  but  also  the  tracking  objectives  are  specified.  For 


a deterministic  environment,  the  objectives  are  defined  as  follows 


Regulation 


For  the  plant  given  by  (2.8)  find  the  control  u.  such  that 


Trackin: 


For  the  plant  given  by  (2.8),  find  the  control  u,  such  that 


where 


and  u^  is  the  reference  input. 

Note  that  the  polynomials  A^(q~^)  are  not  necessarily  the  same  in 
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The  control  objectives  (3.1)  and  (3.2)  can  be  specified  explicitly  using 
an  appropriate  "reference  model"  of  parallel,  series  or  series-parallel 
structure  [8]  and  imposing  that  the  plant-model  error  goes  to  zero.  One 
obtains  in  this  way  a linear  model  following  control  system. 

This  approach  will  then  allow  to  construct  MRAC  when  the  plant  parameters 
are  unknown  or  vary  during  operation.  Examples  of  linear  model  following 
control  schemes  which  allow  to  achieve  the  control  objectives  (3.1)  or  (3.2) 
will  be  given  next  (other  configurations  are  also  possible). 

Regulation 

Consider  the  plant  given  by  (2.10)  and  the  regulation  objective  (3.1). 
Define  the  "series"  reference  model: 

— M . p 0 -i.  tt*  i v 

*k  ■ »k/k-i  ■ < v )yk  • <3-4) 

(The  output  of  the  reference  model  can  be  interpreted  as  the  desired  predicted 
value  based  on  the  measurements  up  to  k-1.) 

The  regulation  objective  (3.1)  will  be  achieved  if  instead  of  (3.1)  one 
considers  the  new  objective: 

yt  - *k  * yk  - >k/k-i  ’ 0 <3-5) 


and  the  resulting  control  is: 

uk-l  " " b“  ^P!kA-1^ 
0 

where: 


(3.6) 


(3.7) 


Tracking 

Consider  a "parallel"  reference  model  whose  output  specifies  the  tracking 
objective: 


The  design  objective  (3.2)  can  be  replaced  by  the  objective 


which  leads  to  the  control 


But,  instead  of  the  design  objective  (3.9),  one  can  consider  the 


following  one 


which  leads  to  the  control 


Note  that  the  "parallel"  reference  model  (3.8)  with  the  control  objective 


(3.11)  can  be  replaced  by  a "series-parallel"  reference  model  of  the  form 


with  the  objective  (3.9)  and  the  corresponding  control  will  be  given  again  by 


The  minimum  variance  control  can  be  interpreted  in  two  ways:  1)  it  is 


the  solution  for  a particular  case  of  L.Q.G.  problem;  2)  it  is  the  solution  in 


order  to  obtain  y.  » v.  which  is  a particular  A.R.M.A.  model 
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This  second  interpretation  allows  to  consider  as  a generalization  of  the 
minimum  variance  control,  the  class  of  linear  stochastic  controls  which  in  a 
given  stochastic  environment  achieve  a control  objective  specified  by  an 
ARMA  process.  The  control  objectives  are  specified  in  the  following  way. 

Regulation 

For  the  plant  and  its  stochastic  environment  given  by  (2.1),  find  a 
control  u^  such  that: 

DCq"1^  - F(q‘1>\  (3.13) 

where: 

D (q*1)  - l-d^"1  - ...  - dnq"°  (3.14) 

F(q"S  - l+fjq'1  + ...  + frq“r  (3.15) 

and  v^  is  the  sequence  of  independent  random  normal  variables  (0,  a)  generating 
the  disturbance  in  (2.1). 

One  recognizes  that  minimum  variance  control  corresponds  to  D(q  *)  * 1, 

F(q  *■)  ■ 1 when  the  delay  is  1 as  in  Eq.  (2.1)  and  to  D(q  *)  * 1 and 
-l  r _•( 

F (q  ) - (1  + I f.q  ) when  the  delay  is  r+1  [7]. 
i-1  1 

Tracking 

For  a plant  given  by: 

*k  ■ FX-1  + boVi  (3-16) 

and  an  A.R.M.A.  process  given  by: 

A°(q*1)xk  - B*(q_l)vk  (3.17) 

where 

B*(q'1)  - 1 + B°(q"1) 


(3.18) 
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and  is  a sequence  of  independent  random  normal  variables  (0,  a),  find  a 


control  u.  such  that: 
k 


D(q’1)(xk-yk)  " F^-1>vk 


(3.19) 


Similar  to  the  deterministic  case,  the  objectives  can  be  specified  using 
"prediction  reference  models"  and  imposing  an  appropriate  behaviour  to  the 
plant-model  error.  This  scheme  will  be  called  "Linear  Stochastic  (ARMA)  Model 
Following  Control"  systems.  Examples  of  such  configurations  will  be  given 
next. 


Regulation 

For  the  plant  and  its  stochastic  environment  (2.1),  assume  that  the 
following  control  objective  should  be  achieved: 


D(i'l)yt  - vk  . 


(3.20) 


Define  a "series"  reference  prediction  model: 

n 


?k/k-i  ‘ Vk-i  * <t^  4i"  >rk 


(3.21) 


then,  the  regulation  objective  (3.20)  will  be  achieved  if  one  considers  the 
new  objective: 


-sM 


yk  ’ yk/k-l  “ Vk  * 


(3.22) 


This  can  be  obtained  with: 

n 


uk-i  " " CC  2 (ai-d1)‘i"i]yk  + ( 2 biq"1)uk-i-l 

K.  i.  Oq  1 J.  X 1*1  1 1 1 


<t=  ■/‘wA.i'i  • 


(3.23) 


-M 


One  recognizes  easily  that  for  d^  • 0,  1 ■ 1,2, ...,n,  y^^.j  * 0 and  one 
obtains  the  minimum  variance  control  for  the  plant  described  by  (2.1).  Note 
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also  that  the  control  (3.23)  can  be  Interpreted  as  one  which  minimizes  the 
variance  of  the  plant-model  error  (min  E^y^-y^/^..^  ^ * 

Tracking 

The  reference  prediction  model  is  given  directly  by  the  AKMA  process 
which  should  be  tracked.  Consider  the  following  two  cases: 
a)  D(q‘1)  - F(q‘1)  - 1. 

In  this  case,  u^  should  be  such  that: 


x,  - y,  - v. 
k ■'k  k 


(3.24) 


which  in  fact  corresponds  to  min  E{(x^-y^)  }.  u^  will  be  computed  such  that 
y,  be  the  best  predictor  in  the  mean  square  of  x.  . But,  from  [7],  it  is  known 


that  the  optimal  predictor  for  x^  will  be: 


0 -i 


m 


0 -i. 


k/k-1  - ^ ”1 


( 2 a4<J  )\  + ( £ b.q  )v. 


k ' ^ r 
K i-1  x 


fk  * 


From  Eqs.  (3.16)  and  (3.25),  one  obtains: 

°k-l  " ' bJJ  ^PMC®k-l  " alq  ,<*k"yk) 

- <t*  bi’"1)(v,k)]  ■ • r0  C'oVl 

* < s,  V >*k  - < s,  ty  XV’k*' 

i-1  i-1 


where  Pu_  is  given  by  Eq.  (3.7). 

MC 

b)  D(q_1)  - A°(q_1)  ; F(q'1)  - 1 


In  this  case,  one  obtains: 


m 


vi  - - r0  ' (X  l>c*Vl><vv]i 


(3.25) 


(3.26) 


(3.27) 


0,-1, 


by  taking  in  account  that  A (q  Mx^-y^)  “ vk 


■I 
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3.3  Duality 

The  duality  holds  also  between  Linear  Model  Following  Control  and  Linear 
Stochastic  (ARMA)  Model  Following  Control. 

For  regulation,  comparing  Eq.  (3.6)  and  Eq.  (3.23),  one  concludes  that 
duality  holds  either  for: 

■ 0 ; d^  - a^  (3.28) 

or 

dt  - 0 ; ci  “ ai  • (3.29) 

For  tracking  comparing  Eq.  (3.10)  with  Eq.  (3.26),  and  Eq.  (3.12)  with 

£ 

Eq.  (3.27),  duality  holds  if  is  replaced  by  u^  in  the  deterministic  case. 

4.  EXPLICIT  AMD  IMPLICIT  M.R.A.C.  AND  STOCHASTIC  S.T.U.R.E. 

When  the  parameters  of  the  plant  (and  those  of  the  disturbance  in  the 
stochastic  case)  are  unknown  or  vary  during  operation  and  adaptive  approach 
should  be  considered  in  order  to  asymptotically  achieve  the  design  objectives. 

The  connections  existing  between  the  techniques  indicated  in  the  title  of 
the  paragraph  have  been  emphasized  to  a certain  extent  only  in  the  last  years 
and  the  reason  is  that  their  original  development  have  been  done  from  different 
points  of  view.  The  M.R.A.C.  techniques  have  been  Initially  developed  for 
deterministic  continuous  time  tracking  problems  and  the  minimum  variance  STURE 
(which  is  a particular  stochastic  STURE)  has  been  initially  developed  for 
stochastic  discrete  time  regulation  problems. 

We  will  examine  next  some  structural  similarities  between  the  various 
schemes  which  in  fact  are  partly  behind  the  connections  which  can  be  established 
between  them. 

A basic  scheme  for  an  explicit  M.R.A.C.  is  given  in  Fig.  4.1.  In  this 
scheme,  a reference  model  specifies  the  objectives  either  for  tracking  or  for 
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regulation  (for  details,  see  [8]  and  [9])  and  the  controller  Is  directly 
adapted  by  the  adaptation  mechanism  which  process  the  plant-model  error. 

A basic  scheme  for  Implicit  M.R.A.C.  Is  given  In  Fig.  4.2.  An  adaptive 
predictor  derived  from  MRAS  techniques  is  used  as  an  intermediate  step  and  the 
parameter  of  the  predictor  is  used  for  updating  the  controller  such  that  the 
controller  plus  the  adapative  predictor  behaves  like  a reference  model  (l.e. 
they  form  an  implicit  reference  model).  It  Is  clear  that  a first  condition  in 
order  that  the  two  schemes  be  equivalent  is  that  the  two  reference  models  be 
the  same  and  this  is  illustrated  in  Fig.  4.3. 

In  this  case,  the  two  schemes  can  be  equivalent  if  the  error  between  the 
plant  and  the  explicit  reference  model  for  the  scheme  of  Fig.  4.1  and  the  error 
between  the  plant  and  the  adaptive  predictor  (the  output  of  the  implicit 
reference  model)  in  Fig.  4.2  will  behave  identically.  Explicit  conditions  for 
this  will  be  given  in  Section  7. 

It  is  also  interesting  to  remark  that  the  scheme  of  Fig.  4.2  uses  an 
extension  of  the  separation  theorem,  i.e.  in  a first  step,  one  designs  a pre- 
dictor of  the  controlled  output  and  in  the  second  step,  one  designs  a control 
for  this  predictor  such  that  the  predictor  output  satisfies  the  design  ob- 
jectives. The  error  with  respect  to  the  initial  objective  being  the  prediction 
error. 

The  basic  scheme  illustrating  the  "STURE"  philosophy  is  given  in  Fig.  4.4. 

One  recognizes  two  steps  which  are  inter- related,  i.e.  1)  on-line  parameter 
estimation;  2)  computation  of  the  controller  from  the  current  parameter  estimates. 
These  two  steps  are  inter-related  because  in  many  cases  by  a convenient  para- 
meterization of  the  prediction  model  used  for  parameter  estimation,  the  computa- 
tion of  the  controlled  is  drastically  simplified.  This  simplification  arises 
when  the  controller  parameters  can  be  explicitly  expressed  by  linear  relations 
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in  terms  of  parameter  estimates.  One  says  that  the  prediction  model  is  para- 
meterized in  terms  of  controller  parameters.  Astroin  and  co-workers  call  this 
a STORE  with  "implicit"  identification.  For  details,  see  [10]. 

However,  for  a better  understanding  of  the  STORE,  one  should  recall  that 
all  the  on-line  parameter  estimation  techniques  are  prediction  error  methods, 
i.e.  they  use  an  adaptive  predictor  which  is  updated  by  an  on-line  identifi- 
cation algorithm  (adaptation  mechanism).  With  this  remark  and  for  the  case  when 
the  controller  parameters  depend  explicitly  and  linearly  on  the  estimated  ones, 
the  STORE  have  always  the  structure  shown  in  Fig.  4.5. 

One  immediately  recognizes  a structural  similarity  with  the  implicit  MRAC 
shown  in  Fig.  4.2.  The  similarity  goes  further  if  we  note  that  in  the  linear 
case  with  constant  parameters,  the  M.V.  control  (see  Section  2)  can  be  obtained 
in  two  steps: 

1)  Optimal  predictor 

2)  Control  of  the  predictor  in  order  to  achieve  a deterministic  objective. 
The  output  of  the  predictor  behaves  in  this  case  as  an  implicit  reference 
prediction  model.  This  strategy  is  then  extended  for  MV-STURE  with  the 
difference  that  the  optimal  linear  predictor  is  replaced  by  an  adaptive  pre- 
dictor. In  this  case,  the  controller  plus  the  adaptive  predictor  form  an  implicit 
reference  prediction  model. 

The  linear  stochastic  (ARMA)  model  following  control  systems  when  the 
plant  parameters  are  unknown  leads  also  to  STORE  configurations  of  the  form 
given  in  Fig.  4.5,  but  as  it  will  be  shown  in  Section  9,  another  configuration 
is  also  possible. 

Few  conments  should  be  made  upon  the  terms  "self- tuning"  and  "adaptive" 
which  are  directly  connected  with  the  nature  of  the  adaptation  gains.  In  the 
"self-tuning"  case,  one  assumes  that  the  plant  parameters  are  constant  but 
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unknown,  which  together  with  the  presence  of  the  stochastic  environment,  leads 
as  for  on-line  identification  of  linear  plants  with  constant  parameters  to  the 
use  of  time-decreasing  adaptation  gains  (least  squares  types  algorithms  or 
stochastic  approximation  types  algorithms).  However  in  order  to  be  able  to 
track  slowly  time  varying  plants,  algorithms  with  time  varying  adaptation  gains 
have  been  Introduced.  An  analysis  of  the  behaviour  of  the  S-STURE  becomes  much 
more  complicated  in  this  case.  In  the  "adaptive"  case,  one  assumes  that  the 
environment  is  deterministic  and  therefore  asymptotic  stability  of  the  full 
system  can  be  assured  using  either  constant,  time -deer easing  or  time-varying 
adaptation  gains.  It  is  assumed  also  that  plant  parameters  are  constant  over 
large  periods  of  time,  but  when  using  constant  or  appropriate  time-varying 
adaptation  gain,  the  adaptive  system  can  react  to  a change  in  parameters 
occurring  at  a random  Instant  which  is  not  the  case  when  using  time-decreasing 
adaptation  gain. 

However  when  M.R.A.C.  are  used  in  a stochastic  environment  convergence  to 
fixed  values  of  controller  parameters  can  be  obtained  (and  analysed)  only  when 
time-decreasing  adaptation  gains  are  used. 

Note  also  that  time -deer easing  and  time-varying  adaptation  gains  provide 
better  adaptation  performance  than  constant  adaptation  gains  even  in  a deter- 
ministic environment  since  they  modify  not  only  the  magnitude  of  the  correction 
at  each  step  but  also  the  direction. 

5.  TOOLS  FOR  ANALYSIS  AND  SYNTHESIS 

Both  M.R.A.C.  and  stochastic  STURE  are  non-linear  time-varying  systems 
and,  therefore,  one  of  the  crucial  points  is  to  examine  their  convergence 
properties  as  time  goes  to  infinity.  This  ismedlately  brings  the  stability 
aspects  in  view.  In  fact,  the  analysis  and  synthesis  of  both  M.R.A.C.  and  STURE 
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involve  two  points,  one  of  algebraic  nature:  does  a linear  controller  which 
can  achieve  the  objectives  exist?  And  this  is  equivalent  to  defining  the 
desired  equilibrium  point  of  the  system,  and  the  second  point  involves  a 
stability  analysis  with  respect  to  this  possible  equilibrium  point. 

However  since  M.R.A.C.  are  of  deterministic  nature  and  S-STURE  are  of 
stochastic  nature  the  stability  concepts  and  tools  of  analysis  will  of  course 
be  different.  Note  also  that  once  stability  concepts  and  appropriate  tools 
for  analysis  being  defined  the  problem  can  be  reversed  into  a design  one  by 
imposing  to  find  an  adaptation  mechanism  which  assures  that  the  equilibrium 
point  have  the  desired  stability  properties. 

The  two  methods  which  will  be  presented  next,  one  for  analysis  of  the 
global  asymptotic  stability  in  a deterministic  environment  and  the  other  one 
for  the  analysis  of  the  w.p.l  convergence  in  a stochastic  environment  have 
been  successfully  used  for  analysis  and  design  of  on-line  identifiers,  adaptive 
observers  and  adaptive  state  estimators  [11],  [12],  [13].  However  when  using 
them  for  the  analysis  and  design  of  MRAC  and  S-STURE,  they  cannot  give  a full 
proof  for  global  convergence  since  the  proof  of  boundness  of  some  variables 
(control  and  plant  output)  requires  an  additional  analysis  [5],  [14].  Never- 
theless recent  work  has  shown  that  adaptation  mechanisms  designed  using  these 
approaches  assure  also  the  boundness  of  the  various  variables  which  imply  that 
global  convergence  is  assured  [5],  [14],  [15].  In  order  to  carry  on  the 
similarity  between  MRAC  and  S-STURE  the  adaptation  algorithm  used  for  MRAC  will 
be  particularized  to  the  form  which  is  Identical  to  that  used  for  S-STURE  (for 
other  adaptation  algorithms,  see  [9],  [16]). 

The  basic  parametric  adaptation  algorithm  considered  throughout  the  paper 
is: 

5<k)  . P(k-i)  + 


(5.1) 
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where  is  in  general  a linear  combination  of  the  "a  posteriori"  generalized 
error  (called  also  "a  posteriori"  plant-model  error)  and  of  its  previous 
values  •••  0^.1  i*  th*  observation  vector  and  ia  the  adapta- 

tion gain  given  by: 

f;1  - + wA  <5-2> 


with  FQ  > 0,  0 < X^k)  < 1;  0 < X2(k)  < 2. 

Note  that  X^(k)  a 1,  X2(k)  = 0 corresponds  to  the  "classical"  constant 
gain  algorithm,  and  X^(k)  * 1,  X2(k)  > 0 corresponds  to  a "time-decreasing" 
adaptation  gain  algorithm.  For  X^(k)  » 1,  X2(k)  • 1,  one  have  a "least  square" 
type  algorithm.  Note  also  that  depending  on  the  choice  of  X^(k)  and  X2(k),  one 
can  obtain  various  laws  for  the  variation  of  the  gain  matrix  Ffc  [8],  [16]. 

in  Eq.  (5.1)  as  it  will  be  shown  next  depends  in  fact  of  p(k)  and 
therefore,  Eq.  (5.1)  cannot  be  directly  used  to  update  £(k).  However  by  an 
appropriate  design  of  the  adaptive  system  can  be  expressed  in  terms  of 
which  is  a linear  combination  of  the  last  "a  priori"  generalized  error  e£ 
(called  also  "a  priori"  plant-model  error  or  "prediction"  error)  and  the 
previous  values  of  *k  (ck_j,  ...  ek_n).  For  the  remainder  of  this  paper,  the 
designs  considered  will  be  such  that  the  relation  between  and  be  of  the 
form: 


1 + *k-lFk-l0k-l 


(5.3) 


For  other  possible  design  leading  to  slightly  different  expressions  for 
vk»  8«e  [8]. 

In  the  context  of  S-STURE,  an  analysis  can  be  made  only  for  the  "decreasing" 
gain  case,  because  the  noise  which  will  enter  in  Eq.  (5.1),  through  vk  (only 
for  a decreasing  gain  F^.^,  p(k)  will  converge  to  a fixed  value). 
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The  analysis  of  the  M.R.A.C.  and  S-STURE  using  a parametric  adaptation 
algorithm  of  the  form  (5.1)  (with  the  restriction  indicated  above)  can  be  done 
using  the  EFR  method  (equivalent  feedback  representation)  and  the  OK  method 
(ordinary  differential  equation)  respectively.  The  basic  ideas  behind  the 
EFR  method  is  to  associate  with  the  equation  of  the  generalized  error  an 
equivalent  feedback  system  representation  which  can  be  partitioned  in  a feed- 
forward linear  time  invariant  block  and  a time-varying  nonlinear  feedback  block 
and  then  the  analysis  of  the  stability  of  this  equivalent  feedback  system  is 
carried  on  using  positivity  and  hyperstability  concepts.  For  details,  see 
[9],  [13],  [16]. 

The  basic  idea  of  the  ODE  method  developed  by  LJung  [12],  [17],  is  to 
associate  with  the  algorithm  (5.1)  an  ordinary  differential  equation  which  will 
describe  the  asymptotic  properties  of  the  algorithm.  Note  also  that  these  tools 
will  allow  also  to  give  answers  to  another  two  problems  of  importance  namely: 
what  will  be  the  behaviour  of  MRAC  in  the  presence  of  stochastic  disturbances 
and  what  will  be  the  behaviour  of  S-STURE  in  a deterministic  environment.  Let's 
state  next  two  basic  results  derived  from  these  two  methods  and  which  are  use- 
ful for  the  analysis  of  MRAC  and  S-STURE,  using  the  parametric  adaptation 
algorithm  (5.1). 

THEOREM  5.1  (E.F.R. ) 

Assume  that  the  parametric  adaptation  algorithm  is  given  by  Eq.  (5.1)  and 
(5.2).  Assume  that  the  following  relation  exists  between  and 

vk  * H(q"l)[p  - p(k)]T0k-1  (5.4) 

where  H(z  S is  a rational  transfer  function  normalized  under  the  form: 


. 1 + h'z’1  + ...  h'z’a 

H(z”;  - » 


1 + hj^z"1  + 


V"9 


• • • 


(5.5) 
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Then: 

lim  vk  * 0 V p(0)  - p,  vQ  (5.6) 

k — «• 

if  the  transfer  function: 

- H(z'1)  - | (5.7) 

is  strictly  positive  real,  where: 

max  \«(k)  £ X < 2 . (5.8) 

0 < k < • * 

For  the  proof  of  this  theorem,  see  [8],  [13]. 

THEOREM  5.2  (O.D.E.)  [12],  [17] 

Assume  that  the  parametric  adaptation  algorithm  is  given  by  Eqs.  (5.1) 
and  (5.2)  with  X^(k)  ■ 1 and  X^(k)  * X^. 

Suppose  that  the  stationary  processes  (^(p)},  [vk(p)}  can  be  defined  for 
all  possible  values  of  pk« 

Assume  that: 

1)  ^k(P)  " H(q"1)0^_1(p)[p*  - p]  + vk  (5.9) 

2a)  vk  is  a sequence  of  independent  normal  random  variables  (0,  a) 
or 

2b)  [^.^(P)}  and  [vk]  are  independent  stationary  stochastic  processes. 

3)  EtVitf).  Vl<P)KP*  - P]  « 0 (5.10) 

has  a unique  solution  p ■ Pj(\_i(P)  " H(9  1)0jt_i(?)* 

Then: 

Prob  [ lim  p(k)  • p*}  ■ 1 (5.11) 

k -•  * 

if  the  transfer  function: 

H' (z"1)  - H(z-1)  - | 


•v 


(5.12) 
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is  strictly  positive  real. 

For  the  proof  of  this  theorem,  see  [13].  Note  the  similarity  between  the 
two  theorems.  Equation  (5.9)  becomes  identical  to  Eq.  (5.4)  when  p(k)  ■ p 
and  ■ 0 and  the  convergence  conditions  are  exactly  the  same  if  the  same 
adaptation  algorithm  is  used. 

Note  also  that  Theorem  5.2  can  be  used  for  the  analysis  of  M.R.A.C.  in 
the  presence  of  stochastic  disturbances.  If  a M.R.A.C.  under  consideration 
verifies  in  a deterministic  environment  Theorem  5.1,  then  v^  in  Eq.  (5.9)  is 
the  image  of  the  disturbance  acting  upon  the  M.R.A.C.  in  the  error  equation 
for  pk  - p. 

Then  w.p.l  convergence  of  p(k)  to  p*  will  be  achieved  if  conditions  2 and 
3 of  Theorem  5.2  are  verified. 

6.  TWO  BASIC  SCHEMES 

We  will  consider  next  an  MV-STURE  as  an  example  of  S-STURE  and  an  example 
of  implicit  MRAC  used  for  regulation.  These  schemes  will  be  used  then  to 
emphasize  the  interrelations  existing  between  Implicit  and  Explicit  MRAC  and 
between  S-STURE  and  M.R.A.C. 

6.1  The  minimum  variance-STURE 

The  equations  describing  the  MV-STURE  are  briefly  reviewed.  For  details, 
see  [10].  The  plant  is  given  by  Eq.  (2.1)  repeated  here  for  convenience: 


>V  ■ pX-i  + boVi  + c<’'1)vk 

(6.1) 

P0  " C*1  •"  *n’  bl  •”  bJ 

(6.2) 

*k-l  " £yk-l  yk-n*  uk-2  uk-m-l^ 

(6.3) 

C(q’1)  - [1-Cjq-1  ...  - cnq-n]  . 

(6.4) 

where: 
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* 
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; 


I ; 
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As  mentioned  in  the  introduction,  bQ  is  supposed  known  and  the  parameter 
T *1 

vector  Pq  as  well  as  C(q  ) are  supposed  ukaova  and  constant  (a  similar  develop- 
ment can  be  done  for  the  case  b^  unknown  and  a basic  delay  superior  to  1).  As 
for  the  linear  case  with  known  parameters  the  polynomials  B(q  1)  and  C(q  l) 
given  in  Eqs.  (2.3)  and  (6.4)  are  supposed  to  have  all  their  zeros  inside  the 
unit  circle. 

The  MV-STURE  algorithm  is  conceptually  obtained  in  two  steps  using  an 
extension  of  the  separation  principle. 

Step  1:  The  adaptive  predictor 


Vk-l  ' ?M»(k-1,0k-l  + boVl 


(6.5) 


where : 


pJyOc)  - [px(k)  ...  pn(k),  6L(k)  ...  6tt(k)] 

* [aL1(k)-c1,  ...  an(k)-cn,  b^(k)  ...  bffl(k)]  . (6.6) 

The  adjustable  parameter  vector  Pjjy(*0  is  updated  using  the  algorithm: 


ta®  ■ ft»(k'l>  + 


^k-l^k-1  o 

1 + 4-iViVi  'k 


(6.7) 


where: 


ek " yk ' K/k-i 

and  F^  ^ is  given  by  (5.2)  with  X^(k)  ■ 1 and  X^(k) 


(6.8) 

Xji  0 < X2  < 2. 


Step  2:  Determine  a control  for  the  adaptive  predictor  (6.5)  (which  will 
be  also  applied  to  the  plant)  such  that  the  deterministic  objective  y^^  * 0 
be  achieved  (as  for  minimum  variance  control  with  known  parameters).  From 
Eq.  (6.5),  one  obtains: 


» 


*1 


I 
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Vl  * - r 

o 


(6.9) 


But,  using  (6.9),  one  has: 


o | 

*k  'u^  - Eq.  (6.9)  “ yk 


(6.10) 


and  the  adaptation  algorithm  (6.7)  becomes: 


PmV00  " PMV(k_1)  + 


c-l0k-l 


T 7k  * 

1 + \-lFk-l*k-l 


(6.11) 


Applying  Theorem  5.2,  one  gets  the  following  result  (see  [12]). 

THEOREM  6.1  (LJung) 

The  MV-STURE  defined  by  Eqs.  (6.1)  through  (6.4)  converges  w.p.l  to  the 
minimum  variance  control,  i.e.: 


Prob  [ lim  ^(k)  - Pmv)  - 1 

k -•  OB 


(6.12) 


where : 


PMV  * Cal"cl  •**  W bl  bm] 


(6.13) 


if  the  discrete  transfer  function: 


H '(z'1) 


C(*_1)  2 


(6.14) 


is  strictly  positive  real  and  the  estimated  parameter  vector  p._,(k)  belongs 

MV 

infinitely  often  to  Dc  defined: 

w 

Ds  - [A(z-1)  • B(z_1)  - A(z‘1)B(z_1)  - 0 ->  |z|  < 1}  (6.15) 


where 


A(z_1)  - 1 - pj^z”1  ...  - paz"n 

B(z-1)  - b + b.z  1 ...  + bz"m  . 
oi  m 


(6.16) 

(6.17) 


6.2  An  Implicit  M.R.A.C. 

As  mentioned  in  Section  4,  the  design  of  an  implicit  M.R.A.C.  is  made  in 
two  steps:  1)  Design  of  an  adaptive  predictor;  2)  Design  of  a control  for  the 
predictor  (which  is  also  applied  to  the  plant)  in  accordance  with  the  regulation 
or  tracking  objectives,  such  that  the  predictor  output  behaves  like  the  output 
of  a reference  model  specifying  the  design  objectives. 

The  process  to  be  controlled  is  described  by: 

yk  ■ Po\-l  + Vk-1  ; * 0 <6-18> 

(the  parameter  Pq  is  supposed  unknown  but  constant.) 

Step  1:  The  adaptive  predictor  ("series- parallel  type"  is  described  by: 


where : 


?wk-i  ‘ ?k  ‘ 5 <k'l,4k-i + Vt-i + c Vi 
?k/k  ■ 5k  ■ pT(k>\-l  + Vk-l  + c\-i 


O AO 

* y l-  ~ y ir 


ek-l  ” ^*k-l  *•*  *k-n^ 

T r t 

c - [-c1  ...  -cn] 


(6.19) 

(6.20) 

(6.21) 

(6.22) 

(6.23) 

(6.24) 


(y£  and  y^  are  also  called  "a  priori"  and  "a  posteriori"  output  respectively 
[8],  [9]). 

From  Eqs.  (6.18),  (6.20)  and  (6.22),  one  obtains: 

•k  ■ l>0  - i><k>Ac-l  - ‘Vl  ■ -V  tp-p(k)]\.!.  (6.25) 

C(s  ) 


Equation  (6.25)  is  of  the  form  of  Eq.  (5.4)  of  Theorem  5.1 
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Then,  if  the  adaptation  algorithm  is  given  by: 


P<k>  * P(k_l)  + Fk-l0k-lek 


(6.26) 


where  is  given  by  (5.2).  (Since  pQ  is  supposed  unknown  and  constant, 
a time-decreasing  adaptation  gain  can  be  used,  i.e.  one  can  choose  as  for 
MV-STURE  X. (k)  - 1 and  X0(k)  - X,;  0 < X,  < 2)  and  if: 


H' (z”1)  - 


1 X 

" 2 

C(z  ) 


(6.27) 


is  strictly  positive  real,  one  concludes  that  lim  e.  » 0. 

k - » K 

Note  also  that  can  be  expressed  in  terms  of  e£,  which  is  the  "a  priori" 
error  or  a prediction  error,  when  using  the  algorithm  (6.26): 


«k  ’ t 

1 + ViVA-i 


(6.28) 


which  introduced  in  (6.26)  gives: 


p(k)  - p(k-l)  + 


Fk-l0k-l 

*E-lPk-l*k-l 


(6.29) 


Step  2:  Computation  of  the  control 

The  convergence  of  y^  to  y^  being  assured  for  all  u^  bounded,  one  chooses 
u^  such  that  the  following  objective  be  achieved: 


W i ■ * 0 


(6.30) 


(i.e.  the  desired  value  of  the  process  output  is  0 and  this  is  specified  by 
the  output  of  the  predictor  which  plays  the  role  of  the  output  of  an  implicit 
reference  model). 


, 

* 


Using  (6.19),  one  obtains 


The  adaptation  algorithm  (6.29)  and  the  control  law  (6.31)  becomes  in  this 


Assuming  I < M sod  since  lim  * 0,  one  has 


and  therefore 


where  |5  (k-1)  is  given  by  Eq.  (6.6) 


the  closed  loop  behaviour  will  be  defined  by 


Observe  that  for  p(k)  - p 
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i.e.  Che  desired  poles  of  Che  closed  loop  sysCems  are  Chose  of  Che  predlcCion 

error  equacion  for  p(k)  * p . Therefore  Chis  scheme  achieves  Che  regulacioa 

T 

objecCive  specified  by  Eq.  (6.38)  and  Che  vecCor  c in  Eq.  (6.19)  is  chosen  in 
accordance  with  Chis  objecCive. 

7.  EQUIVALENT  IMPLICIT  AND  EXPLICIT  M.R.A.C. 

The  possibiliCy  of  obcaining  equivalenc  implicic  and  expliciC  MRAC  has 
been  menCioned  in  Section  4.  Here,  we  will  sCaCe  precisely  Che  equivalence 
becween  Che  Cwo  schemes  and  we  will  give  Che  design  of  an  ExpliciC  MRAC  which 
is  equivalenc  Co  Che  implicic  MRAC  presenCed  in  Seccion  6.2. 

DEFINITION  7.1:  (Equivalence  beCween  ExpliciC  and  Implicic  MRAC):  An 
ExpliciC  MRAC  and  an  Implicic  MRAC  are  equivalenc  if  and  only  if: 

1)  Che  equacions  for  Che  generalized  error  are  idenCical, 

2)  Che  parameCric  adapCaClon  algorithms  are  idenCical,  and 

3)  Che  posicivlcy  conditions  for  global  asymptotic  stability  are  idenCical. 
For  designing  Che  expliciC  MRAC,  which  is  equivalenc  Co  Che  implicic  MRAC 

given  in  Section  6.2,  one  should  consider  a reference  model  which  have  Che  same 
output  as  Che  adaptive  predictor  used  in  Che  scheme,  given  in  Section  6.2  and 
one  should  obtain  Che  same  equation  for  Che  generalized  error. 

One  defines  Che  "a  priori"  and  "a  posteriori"  expliciC  reference  model  by: 

- 0 (7.1) 

^ - [p(k)  - pOt-l)]1^  (7.2) 

(for  details,  see  [8],  [9]). 

One  considers  Che  adaptive  control  law: 

Vi  - - r tf<k-l>\-i + c\.0 


(7.3) 


I 


. 
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for  the  plant  given  by  (6.18)  where: 

o 0 

€k  " yk  • xk 

(7.4) 

#k  * yk  ’ *k 

(7.5) 

T 

ek-l  " £*k-l  *•*  *k-n^ 

(7.6) 

T r i 

c “ •••  “cnJ  • 

(7.7) 

From  Eqs.  (6.18),  (7.2),  (7.3)  and  (7.5),  one  obtaias: 

•k-rtiv'wiVi  <7-8> 

c(q  ) 

which  Is  Idenclcal  co  Eq.  (6.26),  therefore  the  first  requirement  of  Definition 
6.1  is  verified.  But,  Eq.  (7.8)  is  of  the  form  of  Eq.  (5.4)  from  Theorem  5.2. 
Therefore,  using  the  algorithm  (6.26)  for  (6.29)),  lim  e.  ■ 0 if  the  transfer 

k “*  OB  K 

function  of  Eq.  (6.27)  is  strictly  positive  real.  This  verifies  the  third 
requirement  of  Def.  7.2.  Taking  in  account  that  ■ y^,  the  parametric 
adaptation  algorithm  of  Eq.  (6.29)  becomes  identical  to  that  of  Eq.  (6.34)  and 
one  concludes  that  the  Implicit  and  Explicit  MRAC  considered  are  equivalent. 

8.  ASYMPTOTIC  DUALITY  BETWEEN  S-STURE  AND  MRAC 

DEFINITION  8.1:  (Asymptotic  Duality  between  MRAC  and  S-STURE):  An 
Implicit  or  Explicit  MRAC  designed  for  a deterministic  environment  is 
asymptotically  dual  with  respect  to  a stochastic  STURE  designed  for  a stochastic 
environment  if  and  only  if: 

1)  The  adjustable  parameter  vectors  are  updated  by  identical  adaptation 
algorithms  (same  structure,  same  observation  vector  (0)  and  same  generalized 

error  (,k))# 

2)  The  positivity  conditions  for  the  global  asymptotic  stability  of  the 
MRAC  and  for  the  w.p.l  convergence  of  the  S-STURE  are  the  same. 
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i] 


3)  The  control  laws  are  asymptotically  Identical  as  k - ®. 

Remarks 

1)  If  the  control  laws  are  Identical  for  any  k,  they  are  called  dual. 

2)  For  tracking  the  point  3 of  Definition  8.1  should  take  in  account 

as  in  the  linear  case  with  known  parameters  (see  Section  3)  the  change  of  the 
reference  input  in  the  deterministic  case  by  a white  sequence  in  the  stochastic 
case. 

Consider  now  the  MV-STURE  given  in  Section  6.1  and  the  Implicit  MRAC  given 
in  Section  6.2.  One  observes  that  they  verify  the  conditions  of  Definition  8.1 
(Eq.  (6.7)  and  Eq.  (6.29)  are  identical,  Eq.  (6.14)  and  Eq.  (6.27)  are  identical 
and  Eq.  (6.9)  and  Eq.  (6.37)  are  identical). 

Same  conclusions  hold  if  one  compares  the  MV-STURE  given  in  Section  6.1 
and  the  Explicit  MRAC  given  in  Section  7 (as  a consequence  of  the  equivalence 
between  the  Implicit  and  Explicit  MRAC  considered).  These  results  are  summarized 
as  follows: 

THEOREM  8.1:  The  Implicit  MRAC  given  in  Section  6.2  and  the  Explicit 
MRAC  given  in  Section  7 are  equivalent  and  both  are  asymptotically  dual  in  the 
sense  of  Definition  8.1  with  respect  to  the  MV-STURE  given  in  Section  6.1. 

In  fact,  given  the  MV-STURE  configuration  of  Section  6.1,  we  have  con- 
structed the  asymptotically  dual  Implicit  and  Explicit  MRAC  configuration. 

These  configurations  are  new  with  respect  to  the  various  known  MRAC  configurations. 
Their  originality  came  from  the  fact  that  the  generalized  error  is  equal  to  the 
process  output  while  in  all  the  other  configurations  this  is  never  the  case 
(despite  that  they  can  have  the  same  objective  as  it  will  be  shown  in  Section  10). 





s ...  ..?■  -J*  V,  ■ 
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9.  STOCHASTIC  STURE  WITH  EXPLICIT  PREDICTION  REFERENCE  MODELS 

We  heve  seen  in  Section  3 the  connections  between  linear  model  following 
control  and  linear  stochastic  (ARMA)  model  following  control  and  in  particular 
the  fact  that  in  both  cases,  the  control  objectives  can  be  specified  by  a 
reference  model  and  a condition  upon  the  plant-model  error.  We  have  seen  also 
in  Section  4 the  structural  similarities  between  S-STURE  and  Implicit  MRAC.  In 
Sections  7 and  8,  the  equivalence  of  an  Implicit  and  an  Explicit  MRAC  which  are 
both  asymptotically  dual  with  respect  to  MV-STURE  (which  is  a particular  S-STURE) 
have  been  shown. 

The  natural  question  which  comes  up  is:  Does  there  exist  an  equivalent 
realization  of  S-STURE  which  features  structural  similarities  with  explicit 
MRAC?  The  answer  is  yes  and  always  the  diagram  given  in  Fig.  9.1  can  be  filled 
up  if  the  S-STURE  under  consideration  is  of  the  form  where  the  controller  para- 
meters are  linear  explicit  function  of  the  estimated  parameters  of  the  adaptive 
predictor.  This  equivalent  realization  of  S-STURE  uses  an  "explicit  prediction 
reference  model"  and  will  be  illustrated  for  the  case  of  the  MV-STURE  considered 
in  Section  6.1.  However,  in  this  particular  case,  the  MV-STURE  with  "explicit 
prediction  (reference)  model"  will  be  indistinguishable  from  the  MV-STURE  of 
Section  6.1  which  will  be  called  with  Implicit  prediction  (reference)  model, 
because  the  output  of  the  prediction  model  is  always  zero.  The  scheme  is  given 
in  Fig.  9.2  and  the  corresponding  equations  are: 

The  explicit  prediction  model: 

Vk-l  ' ° ' (,-l) 

The  plant  is  given  by  Eq.  (6.1)  and  the  control  is  given  by: 

Vi  - * r [pMv(k*1)0k-i]  • 
o 


(9.2) 
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The  prediction  error  is  defined  as: 


“ y l 


x/k-1 


(9.3) 


and  the  adaptation  algorithm  is  given  by: 


fmr<k) 


5MV(k-1)  + 


Fk-l\-l 


1 + £lFk-l*k-l  k 


PMV(k-1)  + 


Fk-l\-l 


1 + ®k-lFk-l«k-l 


rk  * 


(9.4) 


Similar  to  the  deterministic  case,  one  introduces  the  following  definition 
for  equivalence  in  the  stochastic  case: 

DEFINITION  9.1:  (Equivalence  between  S-STURE  with  implicit  and  explicit 
prediction  models):  A S-STURE  with  I.P.M.  and  a S-STURE  with  E.P.M.  are 
equivalent  if: 

1)  The  equations  for  the  prediction  error  are  identical. 

2)  The  parametric  adaptation  algorithms  are  identical. 

3)  The  possible  convergence  points  are  Identical. 

4)  The  positivity  conditions  for  w.p.l  convergence  (if  they  exist) 
are  identical. 

It  is  obvious  that  the  S-STURE  with  implicit  prediction  model  given  in 
Section  6.1  and  the  S-STURE  with  E.P.M.  given  above  are  equivalent  in  the 
sense  of  Definition  9.1. 

10.  THE  POSITIVITY  PROBLEM  AND  THE  CONVERSE  DUAL  PROBLEM 

The  analysis  of  the  MRAC  scheme  given  in  Section  6.2  has  shown  that  a 
condition  for  global  asymptotic  stability  is  that: 


c 


be  strictly  positive  real,  where  C(z  ) defines  the  desired  poles  of  the  closed 
loop.  This  condition  is  restrictive  (as  for  the  MV-STURE)  since  it  limits 
drastically  the  region  of  the  possible  poles  in  the  z-domain. 

In  the  MRAC  designs,  three  solutions  have  been  considered  in  order  to 
overcome  this  problem: 

1)  Introduction  of  a linear  compensator  acting  on  the  generalized  error 
«k  [8],  [9]. 

2)  Modification  of  the  MRAC  configuration  (i.e.  of  the  reference  model) 

[8]. 

3)  Introduction  of  appropriate  filters  for  generating  the  observation 
vector  0^  [8],  [18]. 

The  first  and  third  solutions  are  to  a certain  extent  similar  since  the 

objective  is  to  introduce  a numerator  in  the  transfer  function  H(z  S appearing 

in  Eq.  (5.4)  (respectively  — r-  in  (10.1))  and  we  will  consider  next  only  the 

C(z_1) 

first  two  solutions. 

The  first  solution  applied  to  the  implicit  MRAC  or  explicit  MRAC  considered 
in  Sections  6.2  and  7 consists  of  the  introduction  of  a linear  compensator 
D(z~S  whose  input  is  *k  and  whose  output  is  v^.  One  defines  the  a priori  and 
a posteriori  processed  generalized  error  as: 

o\-l  <10-2> 

vk  - «k  + dT«k.i  “ D<<!  1)<k  • <10.3) 

becomes : 


The  relation  between  vk  and 


I 

rl 
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and  using  the  adaptation  algorithm: 


pOO  - p(k-l)  + 


Fk-l\-l 

1 + *Vi*k-i 


vk  - p(k-l)  + Fk.^k.iVk  . (10.5) 


The  positivity  condition  of  Theorem  5.1  is  satisfied  if: 


PC?— -2  . A 

cu*1)  2 


(10.6) 


-1, 


is  strictly  positive  real.  Given  a C(z  ) which  is  asymptotically  stable, 
one  can  always  determine  an  appropriate  D(z~^)  in  order  to  satisfy  the 
positivity  condition  through  the  use  of  the  positive  real  lemma  [9]. 

Consider  now  an  explicit  MRAC  where  the  regulation  objectives  C(q  Syk  * 0 
are  specified  by  an  "explicit"  series  parallel  reference  model  with  a priori 
and  a posteriori  output: 

n 

(10.7) 


< ' E,  ciVi 

i*l 


\ m \ + [P(k>  - P(k-l)]T0k_1 


(10.8) 


Note  that  the  a priori  output  of  the  reference  model  gives  the  desired 
prediction  value  of  the  process  output  based  on  previous  output  measurements. 
The  plant  is  given  by: 


yk  “ P0*k-1  + boVl  ; y(0)  * 0 


(10.9) 


and  the  control  is  given  by: 


Vi " • r [pT(k‘D\-i] 

o 


where: 


p(k)  - p(k-D  + 


Fk-i0k-i 


1 + ^.iFk-i\-i  k 


(10.10) 


(10.11) 


: 


r 
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(10.12) 


The  equation  for  the  "a  posterior"  generalized  error  is: 

«k  " £po  " P<k)3T\-i  ciyk-i  ” ^PMV  " p(k)^  ®k-l  (l0*13) 


where  p._,  is  given  by  (6.13). 

MV 

Applying  Theorem  5.1,  one  finds  that  using  the  adaptation  algorithm 

(10.11),  lim  «,  =>  0 without  any  positivity  condition  to  be  satisfied 
k -•  oo  K 

because  in  this  case: 

H'  (z”1)  - 1 - | > 0 . (10.14) 

One  can  also  inmediately  construct  an  implicit  MRAC  equivalent  to  the 
above  explicit  one  and  then  one  can  ask  what  are  the  S-STURE  with  I PM  or  EPM 
which  is  dual  to  the  MRAC  considered? 

The  result  of  this  investigation  leads  to  the  following  S-STURE  with  IPM 
and  E.P.M. 


S-STURE  with  I.P.M. : 

The  process  and  its  environment  is  given  by: 


yk  * p0^k- 1 + boUk-l  + vk 


(10.15) 


where  v^  is  a sequence  of  independent  normal  random  variable  (0,  a).  The 
objective  to  be  asymptotically  achieved  is:  C(q  Sy^  * vk* 

The  adaptive  predictor  is  given  by: 


yk/k-i  * po(k-l)\-i  + boVl 

The  adaptation  algorithm  is  given  by: 

Fk-l0k-l 


pQ(k)  - PQ(k-l)  + 


1 + \-iFk-iVi 


o 

*k  ‘ 


(10.16) 


(10.17) 
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The  control  should  be  such  that  y^/k-1  ■ E Ciyk-1  w^icil  ^ea<^8  t0: 


Vi  ■ - r »o(k-l>  - \ vm1 

o i-1 


(10.18) 


S-STURE  with  E.P.M.: 

The  explicit  prediction  model  will  be  given  by: 

n 

xk/k-l  “ ^ ciyk-i  ‘ 

The  plant  is  given  by  (10.15)  and  the  control  law  by: 

Vl  • - b"  . 

o 

The  adaptation  algorithm  is  given  by: 


P(k) 


p(k-l) 


Fk\-1 

1 + 0k-lFk-l\-l 


(10.19) 


(10.20) 


(10.21) 


Using  Theorem  5.2,  the  both  STURE  converge  w.p.l  to  the  linear  stochastic 
controller  assuring  C(q  Syk  ■ v^  for  a stochastic  disturbance  v^. 

Note  that  in  this  case  the  MRAC  and  S-STURE  are  dual  because  the  control 
laws  are  Identical  for  any  k.  However,  the  main  conclusion  of  this  analysis  is 
that  the  positive  real  condition  can  be  removed  in  the  deterministic  context 
for  achieving  the  same  control  objectives  but  in  the  stochastic  case,  the 
removing  of  the  positivity  condition  correponds  to  a change  of  the  nature  of  the 
disturbance  and  of  the  control  objectives  (or  only  one  of  two). 

A similar  conclusion  holds  when  a linear  compensator  is  introduced  in  the 
S-STURE  given  in  Section  6.1.  Defining  ■ D(q  ^)<^>  this  leads  to  the 
condition  (10.6)  for  w.p.l.  convergence  if  vfc  is  replaced  by  w^  » — v^  and 


-U 


D(q  ) 


the  control  objective  becomes  y^  • w^  (or  D(q  )y^  * v^). 
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11.  THE  NOISE  EFFECT  UPON  M.R.A.C  AND  THE  DUALITY  MRAC -STORE 

One  of  the  important  questions  in  designing  MRAC  is  their  behaviour  in  the 
presence  of  stochastic  disturbances  acting  upon  the  plant.  The  choice  between 
various  possible  configurations  [8]  or  the  development  of  new  configurations 
will  depend  on  their  desired  properties  in  a stochastic  environment. 

The  analysis  of  MRAC  in  a stochastic  environment  can  be  done  using 
Theorem  5.2.  Little  work  has  been  done  in  this  area  [l]  but  work  is  in  progress. 
To  illustrate  this  aspect,  one  considers  the  implicit  MRAC  given  in  Section  6.2 
(or  which  is  equivalent  to  the  Explicit  MRAC  given  in  Section  7). 

The  plant  in  this  case  will  be  described  by: 

>V  ' po4k-i  + l,Vi  + c'<’'1>vk  <ll-l> 

( 

where  C'(q  S is  given  by: 

-1  n -i 

C’(q  L)  - 1 - I c'q  1 (11.2) 

i-1  1 

and  v^  is  a sequence  of  independent  normal  random  variables  (0,  a). 

First  remark  is  that  if  c^  * c^,  the  deterministic  control  assures  in  the 
mean  time  the  minimum  variance  control  and  a straightforward  analysis  show  that 
because  of  the  duality  this  scheme  behaves  exactly  as  the  MV-STURE  of  Section 
6.1  (the  transient  control  signals  depending  on  (p(k)  - p(k-l))  disappears 
in  the  analysis  using  Theorem  5.2  since  p(k)  - p(k-i)  ■ $).  Therefore,  we  will 
consider  next  the  case  c^  i c ^ and  we  will  sketch  briefly  the  analysis  using 
Theorem  5.2. 

With  the  new  Eq.  (11.1),  for  the  plant  and  its  environment,  one  gets: 

T n n 

*k  " Epo  " P<k>3  \-i  + 2 ‘ 2 c'v^  + vk  . 

i»l  i-1 


(11.3) 
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Defining  now  the  stationary  sequences  («k(£)}  and  [0^  ^(p)l  for 
p(k)  * p,  Eq.  (11.3)  becomes: 

Tk<p)  - Cp0  - 'iVi  + vk  • Oi.4) 


But,  for  p(k)  - p: 


e£(p)  - «k(p)  - yk(P) 


(11.5) 


and  Eq.  (11.4)  becomes: 


«k(p>  - yk(p)  • Cpmv  - 5]\.i<p)  - ^ ‘lVi  + \ 


(11.6) 


where  p is  given  by  (6.6).  Adding  and  subtracting  in  the  right-hand  side 
MV 

a _ 

+ Z c!y.  , (p),  one  obtains,  taking  into  account  (11.5): 

— i-1  1 K"1 


ek(p)  - yk(p)  • [p  - p]?k.1(p)  + s cl«k(p) 


ciVi  + vk 


(11.7) 


and  finally: 


1 r * 


where : 


«k(p)  rr  Cp  - p]5it.1(p)  + v 

K C'(q  L)  * L k 


p » [a-.+c.-c',  ...  a_+c  -c',  b.  ...  b ] . 

* U 1 1 tX  n n ' 1 m J 


' V -v  § w<  e e • w 

n n a*  1 m- 


(11.8) 


(11.9) 


Since  is  white,  assuming  that  also  condition  3 of  Theorem  5.2  is 
satisfied  (similar  analysis  as  for  S-STURE  [12]),  one  concludes  that  w.p.l 


convergence  of  p(k)  to  p will  occur  if: 


CU’S  2 


(11.10) 


is  strictly  positive  real  and  one  deduces  from  (6.35)  that  the  control  will  be 
as  k -»  •: 
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Probt  11»  Vl  • - r t^Vl  • = Vk-ll 

k -•  oo  o i*l 


b ^PMV^ 
o 


- I 


where : 


(11.11) 


(pMV)  " CVC1  ' Vcn*  bl  bm]  * 


(U.12) 


Note  that  a bias  appears  with  respect  to  the  desired  control  parameter 
vector  for  the  deterministic  situation  which  Is: 


PMV  " [al‘Cl*  *•*  VCn'  bl  •**  bm^ 


(11.13) 


and  this  bias  will  depend  on  the  difference  c^-c^. 

On  the  other  hand,  the  new  convergence  point  corresponds  to  the  convergence 
point  of  the  dual  S-STURE  for  the  same  stochastic  disturbance. 

One  has  then  the  following  important  conclusion: 

THEOREM  11.1:  A M.R.A.C.  in  the  presence  of  a stochastic  disturbance  of 
the  same  structure  as  that  considered  for  its  dual  S-STURE  will  behave 
asymptotically  as  its  dual  S-STURE. 

The  term  "asymptotically"  comes  from  the  fact  that  in  most  of  the  cases, 
the  control  laws  will  become  only  asymptotically  identical  because  of  the 
transient  adaptation  signals  which  are  used  in  MRAC  (but  this  aspect  will  be 
further  Investigated  next). 

12.  THE  S-STURE  IN  A DETERMINISTIC  ENVIRONMENT  AND  THE  DUALITY  MRAC -S-STURE 

Consider  the  MV-STURE  described  in  Section  1.  Assume  v^  ■ 0,  ■ 0, 

i * l,...,n.  Then,  the  MV-STURE  operates  in  a deterministic  environment  and 
a deterministic  stability  analysis  should  be  considered.  Note  that  for  v^  * 0, 
c ^ ■ 0,  1 * 1 the  plant  and  the  predictor  are  identical  to  that  of  the 
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implicit  MRAC  given  in  Section  6.2  and  since  the  c^  = 0,  the  transient  terms 

in  the  control  law  of  the  implicit  MRAC  disappear.  The  transfer  function 

— - * 1,  and  one  concludes  that  both  schemes  become  equivalent  and  will  be 

C(r  ) 

globally  asymptotically  stable.  However  the  disadvantage  of  the  S-STURE  in 
such  a situation  is  that  all  the  closed  loop  poies  will  be  at  z ■ 0,  and  this 
means  a one  step  response  which  is  in  most  of  the  cases  undesirable  because  of 
the  magnitude  of  the  resulting  control. 

Therefore  in  a certain  way,  the  desired  performance  of  S-STURE  in  a 
deterministic  environment  should  be  specified  by  the  designer  and  this  should 
not  modify  the  behaviour  of  the  S-STURE  in  a stochastic  environment.  From  the 
analysis  carried  on  in  Section  11,  this  can  be  achieved  by  replacing  the  S-STURE 
by  its  dual  M.R.A.C.  and  of  course  since  the  c^  + 0 (which  will  specify  the 
desired  behaviour  in  the  deterministic  environment)  the  transient  adaptation 
terms  in  the  control  law  should  be  added  in  order  to  satisfy  Theorem  S.l. 

13.  A COMBINED  MRAC- S-STURE  SCHEME 

We  will  give  next  a scheme  which  illustrates  how  an  adaptive  scheme  be- 
having as  a MRAC  in  a deterministic  environment  and  a S-STURE  in  a stochastic 
environment  can  be  obtained  (it  summarizes  the  previous  analysis  given  in 
Sections  11  and  12). 

The  plant  in  a deterministic  environment  is  described  by: 

yk‘pk-i  + boVi ; * 0 (13*l> 

and  in  a stochastic  environment  is  described  by: 

(U-2) 

where  v^  is  a sequence  of  independent  normal  random  variables  (0,  a)  and: 


The  adaptive  predictor 


where 


where  the  c.  defines  the  polynomial 


which  specifies  the  desired  performance  in  a deterministic  environment 


The  control  law  is  given  by 


and  the  parametric  adaptation  algorithm  is  given  by 


With  respect  to  the  S-STURE,  we  note  the  introduction  of  the  term  c e. 


in  the  predictor  and  of  the  transient  term  in  the  control  law  which  come  from 


the  deterministic  analysis.  As  simulations  have  shown  the  introduction  of  these 


additional  terms  are  useful  for  speeding  up  the  convergence 
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The  above  scheme  has  the  following  properties: 

1)  In  a deterministic  context,  lim  e,  ■ 0 if 

k -*  «#  K 

1 X 


(13.14) 


is  strictly  positive  real  and  the  desired  closed  loop  poles  are  specified  by 

C (z  1)  - 0. 

2)  In  a stochastic  environment,  this  scheme  converges  w.p.l.  to  the 
minimum  variance  control  if: 


1 

c’Cz"1) 


(13.15) 


is  strictly  positive  real  under  the  assumption  that  p(k)  does  not  leave  the 
domain  allowing  to  define  the  stationary  processes  {0fc_1(p)}  and  fek(p)}  (see 
Section  6.1). 

Of  course,  an  equivalent  explicit  MRAC  can  be  defined  having  the  same 
properties  and  the  same  reasonment  can  be  extended  to  other  configurations. 


14.  CONCLUSIONS 

It  was  shown  in  this  paper  that: 

1)  The  duality  existing  between  linear  stochastic  control  and  linear 
deterministic  control  exists  also  between  the  S-STURE  (where  the  objective  is 
to  have  the  output  of  the  process  described  by  a certain  ARMA  model)  and  the 
MRAC  (where  the  objective  is  to  have  the  output  of  the  process  satisfying  a 
certain  difference  equation). 

2)  The  implicit  and  explicit  MRAC  can  be  equivalent. 

3)  The  S-STURE  can  be  interpreted  as  stochastic  MRAC  where  the  reference 
model  (implicit  or  explicit)  is  replaced  by  prediction  models  (explicit  or 
implicit). 
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4)  The  current  used  realization  of  S-STURE  are  of  the  type  using  implicit 
prediction  reference  models  but  equivalent  S-STURE  with  explicit  prediction 
reference  models  can  be  defined. 

5)  The  duality  properties  of  S-STURE  and  MRAC  have  been  exploited  for  the 
analysis  of  MRAC  in  a stochastic  environment  and  of  S-STURE  in  a deterministic 
environment. 

6)  The  duality  analysis  has  led  to  a new  configuration  of  MRAC  assuring 
the  same  regulation  objectives  as  known  MRAC  configurations  but  which  have  a 
different  behaviour  in  a stochastic  environment. 

7)  A method  for  constructing  an  adaptive  scheme  which  behaves  as  a given 
MRAC  in  a deterministic  environment  and  as  a given  S-STURE  in  a stochastic 
environment  has  been  indicated  and  Illustrated  by  an  example. 
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STOCHASTIC  ADAPTIVE  CONTROL  OVERVIEW 

Yaakov  Bar-Shalom 
Dept,  of  Electrical  Engineering 
and  Computer  Science 
University  of  Connecticut 
Storrs,  Connecticut  06268 

1.  Introduction 

This  paper  presents  an  overview  of  the  area  of  stochastic  control  with 
emphasis  on  its  usefulness  for  adaptive  control.  First  the  basic  assumptions 
related  to  probabilistic  modeling  are  presented  in  Section  2.  The  Bayesian 
approach  of  extremizing  the  expected  value  of  a performance  index  (in  general 
minimizing  a loss  function)  is  discussed  in  detail.  The  concept  of  learning 
is  introduced  from  an  intuitive  point  of  view.  Section  3 deals  with  the 
extremization  of  the  performance  index,  which  is  to  be  done  according  to  the 
Principle  of  Optimality  [B6],  The  concept  of  preposterior  analysis,  known  in 
the  Operations  Research  literature  [Rl],  is  shown  to  be  an  immediate  con- 
sequence of  the  Stochastic  Dynamic  Programming  equation  which  follows  from  the 
Principle  of  Optimality.  A classification  of  stochastic  control  laws  is  pre- 
sented that  points  out  the  main  feature  of  an  actively  adaptive  control 
algorithm.  In  such  a case  the  control  can  be  used  for  "active  (control  aided) 
Information  gathering"  to  speed  up  the  adaptation  process.  This  is  possible 
when  the  control  has  the  so-called  "dual  effect,"  first  pointed  out  in  [FI] 
and  rigorously  defined  in  [Bl].  It  is  pointed  out  that  the  dual  effect  of  the 

* 

control  can  be  used  not  only  when  there  are  unknown  parameters,  but  also  in 
their  absence,  when  it  can  enhance  the  state  estimation. 

Section  4 discusses  a number  of  stochastic  adaptive  control  algorithms: 
the  Heuristic  Certainty  Equivalence  Approach,  the  Self-Tuning  Regulator,  the 
Multiple  Model  Weighted  (partitioned)  Adaptive  Control  and  a Closed-Loop  Dual 
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Control.  It  is  shown  how  a decomposition  of  the  optimum  cost  obtained  in  the 
Dual  Control  algorithm  by  a suitable  approximation  of  the  Dynamic  Programming 
points  out  explicitly  the  Caution  and  Probing  effects  caused  by  the  uncertainty 
in  the  problem.  The  application  of  the  dual  control  method  to  a missile 
guidance  problem  with  no  unknown  parameters  but  with  nonlinear  structure 
illustrates  how  the  control  can  be  successfully  used  to  enhance  estimation. 

2.0  The  Basic  Modeling  Assumptions  in  Stochastic  Control 

In  stochastic  control,  modeling  of  the  uncertainty  is  done  as  follows. 
Imperfect  information  is  summarized  in  probabilistic  form: 

(i)  random  variables  - unknown  parameters  or  state 

(ii)  random  processes  or  sequences  in  discrete-time  disturbances 
The  system  equations  are 

x(k+l)  - fk[x(k),  u(k),  0,  v(k) ] (2.1) 

y(k)  - hk[x(k),  w(k)]  (2.2) 

where 

x(k)  - state  vector  at  time  k 

0 - unknown  parameters  with  a prior  pdf 

u(k)  - control  (decision  variable) 

v(k)  - state  equation  disturbance  (process  noise) 

y(k)  - measurement 

w(k)  - measurement  noise. 

For  example,  a linear  system  with  unknown  parameters  is  given  by 

x (k+1 ) - A(0)x(k)  + B(0)u(k)  + v(k)  . (2.3) 

In  the  Bayesian  approach  the  goal  is  to  minimize  the  expected  value  of  a 
loss  function.  In  order  to  be  able  to  obtain  the  expected  value  of  the  loss 
function  every  variable  this  function  depends  upon  must  be 


1 


n 
■ 1 

I i 

K j 
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(i)  deterministic  (i.e.,  known  perfectly),  or 

(ii)  random  - a pdf  has  to  be  attached  to  all  the  random  variables  or 
processes  entering  into  the  description  of  the  system. 

There  are  other  approaches  (less  comnon)  like  the  minimax  [S2]  and  worst 
distribution. 

2.1  The  Bayesian  Approach  for  Discrete  Time  Stochastic  Control 

In  this  approach  one  considers  the  dynamic  model  of  system  given  by 
x(k+l)  - ffc[x(k),  u(k) , v(k) ] k • 0,1,...  (2.4) 

where  unknown  parameters  are  included  in  the  state  vector  - they  might  be  time- 
varying. 

The  information  at  the  start  of  the  process  consists  of  the  joint  pdf  of 
the  initial  state  and  the  sequence  of  disturbances  (process  noise). 

The  cost  function  is 


N-l 


C(0,XN,UN_1)  - Cn[x(N)]  + Z Cjx(k),  u(k)] 


k-0 


where 


XN  - £x(k))k  ; II*"1  - (u(k)}£J  . 


(2.5) 


(2.6) 


Remarks : 

1.  In  some  problems  the  disturbance  might  also  enter  into  the  cost. 

2.  The  terminal  time  can  be 

(i)  fixed 

(ii)  a random  variable  (depending  on  the  state) 

(iii)  a decision  variable. 

The  expected  cost  is 

J - E[C]  (2.7) 


s yfcWPfJF 


- 


— e — rr 
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and  our  problem  is 


min  J 
yN-i 


(2.8) 


Remark: 

The  minimization  of  the  expected  cost  implies  that  we  want  to  find  the 
optimal  policy 

(i)  over  all  possible  initial  conditions. 

(ii)  over  all  possible  values  of  the  unknown  parameters. 

(lii)  over  all  possible  disturbance  sequences. 


2.2  The  Concept  of  Learning 

If  the  system  to  be  controlled  has  some  unknown  parameters  this  initial 
uncertainty  can  be  modelled  by  a prior  pdf  p(9|l^>. 

The  initial  control  u(0)  will  account  for  the  fact  that  it  is  applied  to 
a system  with  parameter  0 "drawn"  from  the  prior  distribution. 

If  the  parameter  0 is  time- Invariant  one  can  reduce  the  initial  uncertainty 
about  its  true  value  in  the  course  of  the  control  process  - the  controller  can 
"learn"  it. 

Thus,  as  new  information  is  gathered  via  feedback,  the  controller  can 
adapt  Itself  to  the  particular  system  it  is  controlling. 

There  are  some  fundamental  questions  related  to  this  concept  of  learning: 

(i)  How  much  is  the  performance  degradation  because  of  the  parameter 
uncertainty? 

(ii)  Can  the  uncertainty  be  reduced  during  the  control  process? 

(lii)  If  the  uncertainty  can  be  reduced,  can  the  control  be  used  to 

reduce  faster  the  uncertainty? 


F 
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3.0  The  Principle  of  Optimality  for  Stochastic  Problems  and  the 
Stochastic  Dynamic  Programming 

The  basic  tool  to  solve  stochastic  control  problems  is  the  Principle  of 
Optimality:  at  any  time,  whatever  the  present  information  set  and  past 
decisions,  the  remaining  decisions  must  constitute  an  optimal  policy  with 
regard  to  the  present  information  set. 

In  the  deterministic  case,  the  state  summarizes  the  past.  In  the 
stochastic  case,  the  information  set  is  what  the  controller  knows  about  the 
system: 

i - (y  , u j - £i  , yk,  uk-1}  . (3.1) 

The  problem  is  to  find 

min  E[C(0,  XN,  U**"1)  |l°]  £ J*(0,  1°)  (3.2) 

u"-1 

with 

uk  " uk(lk)  * 0-3) 

From  the  Principle  of  Optimality  the  last  decision  must  be  optimal  with 
regard  to  the  information  state  available  when  it  has  to  be  computed 

min  E(C  l^-1)  (3.4) 

Vi 

where  C is  the  cost  for  the  entire  problem. 

The  next  to  the  last  decision  u^ 

(i)  must  be  optimal  w.r.t.  I1*-*,  and 

N- 1 

(ii)  is  made  knowing  that  wiH  be  optimal  w.r.t.  I , i.e.,  it 

is  obtained  from 

min  E[min  E(C  |lN‘l)  |lN“2]  . (3.5) 

V2  Vl 


- 


■ 
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Note  that  the  outside  averaging  is  over  y^_ ^ . 

Thus  the  optimal  expected  cost  for  the  N-step  problem  is  obtained  from  a 
sequence  of  nested  expectations  and  minimizations 

J*(0,  1°)  » min  E{...  minE[min  E(C  |lN_1)  |IN"2]  ...  |l°}  . (3.6) 

Uo  “N-2  “n-2 

The  general  stochastic  dynamic  programming  equation  is,  for  an  additive  cost, 

if  If  if  |c4.1  I, 

J (k,  I ) - min  E[Ck(xk,  uR)  + J (k+1,  I ) |lK]  (3.7) 

\ 

with  end  condition 

J*(N,  Ik)  - E[Cn(xn)  |IN]  . (3.8) 

Note  that 

E[j*(k+1,  Ik+l)Hk] 

• J f(yk+i‘yk*““yo)p(yk+iiyk yo)dyk+i  (3-9> 

i.e.,  at  each  iteration  one  has  to  average  over  the  next  observation. 

Thus  the  optimal  control  depends  on 

(i)  the  current  information  u^  ■ uk(I  ). 

(ii)  the  prior  statistical  description  of  the  future  posterior 
information 

p(yj+l  |IJ)  J > k • 

This  Is  preposterior  analysis  [Rl]  which  can  be  paraphrased  as  "know  how 
to  use  what  you  know  as  well  as  what  you  know  about  what  you  shall  know,"  and 
is  a consequence  of  the  Principle  of  Optimality. 
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3.1  Types  of  Algorithms  in  Stochastic  Control 

The  classification  presented  below  can  be  made  for  stochastic  control 
algorithms. 


[ 


^^v^eatures 

Types 

of  algorithm^^ 

Utilization 
of  real-time 
observations 

Utilization  of  the 
statistical  description 
of  future  observations 

Open- loop  (OL) 

HI  H 

X 

Closed-loop  (CL) 

X 

X 

A CL  algorithm  "knows"  that  the  loop  will  stay  closed  throughout  the 
process  (F  + PPA)  (feedback  4-  preposterior  analysis). 

An  OL  algorithm  is  never  optimal  for  a stochastic  problem. 

The  optimum  is  in  general  of  the  CL  type  with  some  exceptions  when  it  is 
of  the  F type. 

The  Open- loop-optimal  feedback  (OLOF)  policy 

(i)  computes  the  control  under  the  assumption  that  no  future 
observations  will  be  available  (OL)  but 

(ii)  when  observations  are  made  they  are  utilized  by  the  controller  to 
update  its  information  about  the  system. 

This  controller  belongs  to  the  F class  according  to  the  above  classification. 

The  m-measurement-optlmal  feedback  policy  computes  the  control  assuming 
measurements  will  be  available  only  at  the  next  m sampling  times 
(m-0  ->  OLOF). 

The  usefulness  of  the  F/CL  distinction  is  in  the  following.  When  the 
optimum  stochastic  control  is  not  known  for  a class  of  problems  one  can  use 
auboptlmal  algorithms  of  the  feedback  type  or  closed- loop  type. 


1 

i 


mm 


92 

To  be  as  close  as  possible  to  the  optimum,  it  is  important  to  realize 
the  distinction  between  F and  CL  and  to  be  able  to  obtain  an  approximation  of 
the  stochastic  dynamic  programming  that  has  the  CL  property. 

3.2  The  Control's  Dual  Effect 


If  the  uncertainty  of  the  state  (which  includes  possibly  unknown  system 
parameters)  of  a stochastic  system  depends  on  past  control  values,  the  control 
is  said  to  have  a dual  effect  [FI,  Bl]. 

The  definition  of  the  dual  effect  is  as  follows:  The  Information  set  at 
time  k is 

Ik  - [Yk,  U11’1}  . (3.10) 

The  state  estimate  (conditional  mean)  is 

ik|k-E(xt|Xk)  • O.H) 

The  covariance  of  the  state  at  time  k is 

\\km  E^xk  " *k|k)(xk  " *k  |k^ ' I ^ • (3*12) 

k-1 

Then,  if  ^ does  not  depend  on  U the  control  has  no  dual  effect  (of  second 
order)  - the  control  is  neutral. 

The  implications  of  the  dual  effect  are  as  follows: 

1.  Active  Information  Gathering  (Probing) 

* If  the  control  has  a dual  effect,  such  that  it  can  reduce  some 
uncertainty,  this  might  be  used  to  improve  the  overall  performance. 

* Only  a closed  loop  control,  by  anticipating  future  feedback, 
can  assess  the  "value  of  future  information"  and  do  a tradeoff 
between 

(i)  control  action 

(11)  information  gathering  to  improve  the  accuracy  of  subsequent 
control  actions. 
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2.  Caution 

• Due  to  the  Inherent  uncertainties  the  controller  has  to  be 
"cautious"  not  to  increase  the  effect  of  the  existing  un- 
certainties on  the  cost. 


3.3  Adaptive  and  Dual  Control 

The  question  of  what  is  Adaptive  Stochastic  Control  as  discussed  at  the 
1976  CDC  [B2]  reflected  the  following  points  of  view: 

1.  The  controller's  actions  are  based  on  a model  of  the  system  that  is 
updated  in  real  time  (because  Initial  information  is  poor  or  system 
changes).  This  has  a hierarchical  structure  of  feedback 

(i)  lower  level:  feedback  control  based  upon  current  model 

(ii)  higher  level:  feedback  is  used  to  update  the  model. 

2.  One  is  faced  with  a nonlinear  stochastic  control  problem  which  has 

to  be  approximated.  Adaptive  control  is  a method  of  approach  for  the 
control  of  systems  when  the  exact  formula  '.ion  is  too  complex. 

A learning  system  is  one  which,  while  operating  in  a stochastic  environ- 
ment, can  reduce  the  uncertainty  in  its  description  as  the  process  evolves 
[SI].  This  is  a similar  definition  to  the  first  one  above. 

Stochastic  adaptive  control  can  be  classified  as  follows: 

1.  Passively  adaptive  control  where  the  controller  learns  about  the 
system  but  does  not  anticipate  subsequent  learning  from  future 
feedback.  Learning  is  therefore  accidental  (passive)  - from  past 
"mistakes."  Such  a controller  belongs  to  the  feedback  class. 

2.  Actively  adaptive  control  - the  controller  learns  about  the  system  and 
anticipates  subsequent  learning  from  future  feedback.  Then  learning 
is  enhanced  by  use  of  the  dual  effect  - the  controller  experiments 


I 
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(probes)  to  improve  the  accuracy  of  information  about  the  model. 

Such  a controller  belongs  to  the  closed-loop  class. 

3.4  Dual  Effect  of  the  Control  and  General  Nonlinear  Problems 

The  dual  effect,  if  accounted  for  by  an  adaptive  controller,  can  make  it 
into  actively  adaptive.  The  main  class  of  problems  where  this  approach  can  be 
used  is  the  control  of  linear  systems  with  unknown  parameters. 

In  other  problems,  e.g.,  linear  systems  (without  parameter  uncertainties) 
and  with  nonlinear  measurement  the  control  has  in  general  a dual  effect.  In 
such  a case  one  cannot  talk  about  an  adaptive  control  but  the  control  can  still 
enhance  the  state  estimation  accuracy  [T3,  Cl]. 

Such  a problem  is  encountered  in  homing  missile  guidance  - a dual  control 
can  then: 

(i)  improve  performance  for  given  sensor  accuracy,  or 

(ii)  lower  sensor  accuracy  requirement  for  given  performance. 

4.0  Some  Adaptive  Stochastic  Control  Algorithms 

4.1  The  Heuristic  Certainty  Equivalence  Approach 

A linear  system  with  unknown  parameters  is  considered 

*k+l  " Fk<e)xk  + Gk<e)uk  + Vk  (4>1) 

with  linear  observations 

?k  * Vk  + wk  • <4-2) 

At  time  k one  has  x^  ^ and  9^.  A common  (and  approximate)  method  of  obtaining 
these  estimates  is  via  the  Extended  Kalman  Filter  (EKF). 

The  control  algorithm  consists  of  the  following.  Using  the  parameter 
estimate  at  t^  one  has: 


With  this  one  computes  the  gain  from  the  standard  LQ  problem  as  if 


All  the  uncertainties  in  F,  G are  ignored  (HCE).  The  control 


is  then  applied.  At  tfc+1  a new  estimate  of  the  parameters  0^^  is  obtained 
and  the  procedure  is  repeated. 

A recent  study  [C2]  showed  stability  for  an  AKMAX  system  with  unknown 
parameters  controlled  by  such  an  algorithm  when  the  parameter  identification 


was  done  via  stochastic  approximation 


4.2  Self-Tuning  Regulator  (STUftE 


An  input-output  model  with  white  noise  e.  is  considered 


The  cost  criterion  is  minimum  variance 


In  the  special  case,  where  C » 1 


The  optimal  minimum  variance  control  (with  known  parameters)  is 


* 


The  adaptive  (self-tuning)  algorithm  for  unknown  parameters  [Al] 


(i)  estimates  the  parameters  (using,  e.g.,  least  squares)  and 


(ii)  uses  same  controller  with  estimated  parameters  in  place  of  the 


The  STURE  is  passively  adaptive  and  belongs  to  the  F class.  The  optimal 
policy  is  of  the  CL  type  since  the  control  has  a dual  effect.  Several  more 
general  versions  are  available  as  well  as  convergence  results  [Ll].  A number 
of  successful  applications  to  practical  problems  have  been  reported  [A2]. 


4.3  Multiple-Model  Weighted  (Partitioned)  Adaptive  Control 


H(Q)x.  + w 


The  parameter  vector  belongs  to  a finite  set  ("set  of  models").  The  initial 


information  is 


For  known  parameters,  the  optimal  control  is 


V0)  “ - Lk(0)*k|k(0) 


The  controller  in  this  approach  [Dl]  is  a weighted  sum  of  the  model -optimal 


controllers 


Note  that  thia  is  not  optimal  and  only  passively  adaptive.  The  recursive 
updating  of  the  parameter  pdf  is  done  using  Bayes’  rule  as  follows: 


k-1 


Plik-F[9.  0t|I  ] 


k,  p(yk|i  ,ei)pt.k-i 


4.4  A Closed-Loop  (Dual)  Control  Algorithm 

The  stochastic  dynamic  programming  equation 

•Jk  If  4>  1 m 

J (k,IK)  - min  E{C.  [x(k),  u(k)]  + J (k+1,  |lK+i) |lK}  (4.16) 

u K 

is  approximated  in  this  approach  as  follows  [Tl,  T2,  B3]. 

Ir 

* The  full  information  vector  I is  replaced  by  an  approximate 
information  state 

Pk  - (x(k|k)f  E(k|k)}  . (4.17) 

The  vector  x is  the  "proper"  state  augmented  by  the  system’s  unknown 
parameters,  if  any.  An  estimator  like  the  EKF  can  be  used  to  generate 
this  information  state. 

The  following  search  procedure  is  used  to  find  the  control  at  time  k: 

* An  arbitrary  control  u^  is  assumed. 

* This  control  yields  a predicted  state  x[k+l  jk;u(k) ] £ Xg(k+1). 

* The  resulting  predicted  state  x^(k+l)  is  taken  as  the  initial  condition 
of  a "nominal"  trajectory  Xq(J),  j • k+l,...,N  generated  with  a sequence 
of  nominal  controls  Uq(J),  j ■ k+l,...,N. 

* A perturbation  analysis  via  second  order  expansion  is  carried  out  about 
the  nominal  trajectory  to  capture  the  stochastic  effects. 
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* The  expected  cost  corresponding  to  this  control  is  then  evaluated 
using  a set  o£  recursions. 

* The  procedure  is  repeated  to  £ind  the  value  of  the  control  that  yields 
the  minimum  o£  the  expected  cost. 

The  resulting  control  depends  on 

* Current  estimate  of  the  (augmented)  state. 

* Current  state  uncertainty. 

* Future  state  uncertainties  as  anticipated:  The  covariance  of  the  state 
is  precomputed  along  the  nominal  trajectory  via  EKF.  Note  that  this 
nominal  trajectory  depends  on  the  current  control  u(k).  Thus,  if  the 
control  has  the  dual  effect,  its  effect  on  the  quality  of  future 
information,  £(j|j),  j > k,  is  automatically  incorporated  in  the 
decision  procedure. 

This  algorithm  has  been  used  for 

* Linear  systems  with  unknown  parameters  (actively  adaptive  control). 

* Nonlinear  systems  (with  no  unknown  parameters)  where  the  control  can 
enhance  the  estimation. 

The  algorithm  consists  of  the  following 

uCL(k)  - arg  min  JCL(k)  . (4.18) 

A very  useful  decomposition  of  the  (closed- loop)  approximation  of  the  optimal) 

cost  can  be  obtained  [B4,  B5 ] : 

JCL(k)  - JD(k)  + Jc(k)  + Jp(k)  . (4.19) 

The  deterministic  component  of  the  cost  is 

Vk)  A + Vk+D  + Y0<k+D  (4.20) 


and  the  stochastic  components  of  the  cost  are 
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Jc<»0  bj  trCK^k+imk+ljk)] 

1 N'1 

+ 4 2 tr[K  (j+l)V  (j)]  (4.21) 

j-K-1 

1 N_l 

Jp(k)^4  2 tr[A  Q(j)2U(J|J)]  (4.22) 

j-k+1 


Aq  - obtained  from  some  recursions. 

- covariance  of  the  augmented  state  evaluated  along  the  nominal 
trajectory. 

- covariance  of  the  process  noise. 


deterministic  component  consists  of 
0jc[u(k)  3 “ cost  of  control  at  time  k. 

(ii)  Cg(k+1)  - cost  incurred  along  the  nominal  trajectory  from  k+1  to 

N;  all  uncertainties  are  ignored  (HCE). 

The  caution  component  consists  of 

(i)  -j  tr[Kg(k+l)2(k+l |k]  - cost  due  to  the  uncertainty  in  the  initial 

condition  Xg(k+1)  of  the  nominal  trajectory.  This  is  a mapping  of 

the  current  uncertainty  - its  effect  on  the  cost. 

1 N_1 

(ii)  E tr[Kn(j+l)V  (j)]  - cost  due  to  the  disturbances  in  the 

L j-k+1  u v 

dynamic  equation  (process  noise) . 

The  caution  component  represents  the  effects  of  the  existing  uncertainties  on 
the  cost.  The  weighting  are,  however,  depending  on  the  choice  of  the  current 

; 

control  u(k). 

The  probing  component  is 
1 N_1 

•z  E tr[An(j)L«(j  |j)]  - weighted  sum  of  the  future  uncertainties 
* J-k+1  0 ^ 

(state  covariances).  These  uncertainties  depend  on  the  current 


where 

V 

■ 

V 

v 

I The 

I 

(i) 


V 
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control  If  it  has  the  dual  effect.  The  weighting  matrix  Aq 
reflects  the  "value  of  future  information." 

Remarks : 

1.  The  caution  component  tends  (in  general,  but  not  always)  to  reduce 
the  value  of  the  control:  larger  control  values  might  increase  the 
effect  of  the  uncertainties  on  the  cost.  The  control  will  be 
"cautious"  due  to  the  uncertainty. 

2.  In  some  problems  it  pays  off  for  the  control  to  probe  in  order  to 


reduce  the  uncertainties  about  the  system. 

3.  A compromise  between  the  control  action,  the  need  for  caution  and 
ttye  desirability  of  probing  has  to  be  made. 

4.  Probing  and  caution  are  usually,  but  not  always,  conflicting. 

Based  on  the  relative  magnitude  of  the  three  cost  components  one  has 

three  major  classes  of  stochastic  control  problems:  If  the  uncertainty 
dominates  the  problem  then  one  can  distinguish  two  cases 

1.  The  caution  component  (Jc>  dominates.  Then,  since  this  is  "un- 
controllable" uncertainty,  one  has  a highly  uncertain  model  which 
cannot  be  improved  in  the  course  of  the  control  period. 

2.  The  probing  component  (Jp)  dominates.  Then,  with  the  dual  effect  of 
the  control  one  can  reduce  the  uncertainty  of  the  model  - thus  the 
model,  while  uncertain  at  the  beginning,  might  prove  to  be  ultimately 
adequate  for  the  control  problem  under  consideration. 

A third  case  occurs  when: 

3.  The  deterministic  component  of  the  cost,  J^,  dominates:  then  the  para- 
meter uncertainties  are  of  no  significant  consequence.  This  is  the 
most  desirable  situation  because  then  we  can  use  CE  (least  expensive) 
control  algorithm  with  good  performance.  However,  only  the  stochastic 
control  approach  can  tell  us  that  fact. 
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4.5  Dual  Control  for  a Missile  Guidance  Problem 


The  missile  is  modeled  by  point  mass  kinematics  with  lateral  acceleration 
?--Tlu  ij  » 4 u (4*23) 

which  leads  to  the  £ollowing  nonlinear  stochastic  plant  equation 

x(t)  - f[x(t)u(t)]  + v (4.24) 

where 

f[x(t)u(t)]  =»  [x2(t)  -x4(t)u(t)  x4(t)  x3(t)u(t)]'  . (4.25) 

The  measurement  consists  of  angle  only 

y(tk)  - tan-1  + w(tfc)  . (4.26) 

The  goal  of  the  guidance  is  to  hit  the  origin  with  noisy  Initial  information 
about  own  initial  location  and  velocity. 


There  are  no  unknown  parameters  in  this  problem. 

The  main  problem  is  the  (nonlinear)  estimation  of  the  state,  in 
particular  the  range  to  the  target. 

Under  straight  flight  conditions  the  on-board  angle-only  sensor  will 
provide  little  Information  about  the  range. 


Remarks : 
1. 
2. 


i 
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The  trajectories  with  HCE  vs.  dual  control  [T3]  turned  out  to  be  as  illustrated 
below: 


i 


The  dual  controller  suggests  deviating  on  both  sides  to  obtain  range 
information  from  the  angle-only  sensor  by  varying  the  geometry  of  the  problem 
during  the  flight. 

Once  this  observation  is  made,  an  off-line  optimization  algorithm  has 
been  used  to  obtain  the  divert  maneuver  that  minimizes  the  terminal  miss 
distance  [Cl]. 

Usefulness  of  these  results  is  in  the  following: 

1.  Improvement  of  miss  distance  for  given  sensor  accuracy,  or 

2.  Lowering  of  sensor  accuracy  requirements  for  given  miss  distance. 

5.  Conclusions 

Wonham  at  the  1968  JACC  stated  the  following:  In  the  case  of  (stochastic) 
feedback  controls  the  general  conclusion  is  that  only  marginal  improvement  can 
be  obtained  (over  a controller  ignoring  the  stochastic  features),  unless  the 
disturbance  level  is  very  high;  in  this  case  the  fractional  improvement  may  be 
large  but  the  system  is  useless  anyway. 
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f: 


Recent  results  show  that  in  some  problems  adaptation  and,  in  particular, 

active  adaptation  can  yield  performance  close  to  the  lower  bound  (when  there 

is  no  uncertainty).  This  class  of  problems  is  characterized  by  dominance  of 

the  probing  term  in  the  cost,  which  can  be  reduced  by  learning. 
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CObMENTS  ON  ADAPTIVE  AND  ROBUST  CONTROL 
C.  A.  Harvey 
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Minneapolis,  MN  55413 


ADAPTIVE  CONTROL 

I believe  the  following  basic  questions  must  be  answered  when  a flight 

control  application  of  adaptive  control  is  contemplated. 

Is  the  synthesis  technology  for  adaptive  control  adequately 
developed? 

Is  adaptive  control  necessary  or  highly  advantageous? 

These  questions  can  be  refined.  For  example,  "adequately  developed"  can  be 
quantified  in  terms  such  as  the  skill  required  of  the  designer  and  the  cost  of 
the  synthesis  in  computer  and  engineering  hours.  Similarly,  "highly  advan- 
tageous" can  be  quantified  and  "necessary"  can  be  qualified  in  terms  of  system 
performance  and  system  cost. 

In  the  NASA  flight  control  research  program  in  digital  fly-by-wire 
technology  it  was  demonstrated  that  some  adaptive  control  technology  is 
adequately  developed  for  synthesizing  an  adaptive  conmand  augmentation  system 
for  an  F-8C  aircraft.  But  the  performance  requirements  for  the  F-8C  are 
readily  met  with  air-data- scheduled  (nonadaptive)  control  laws.  Thus  the  major 
benefit  provided  by  adaptive  control  is  that  the  need  for  air-data  sensors 
could  be  eliminated  which  would  be  advantageous  from  redundancy  considerations. 
I believe  that  this  example  is  typical  and  that  similar  answers  would  apply  to 
future  high  performance  aircraft. 

A flight  control  application  in  which  adaptive  control  seems  to  be 
necessary  or  highly  advantageous  is  the  active  control  of  wing-store  flutter. 
The  motivation  for  this  application  is  that  fighter  aircraft  are  required  to 
carry  many  different  combinations  of  external  stores  to  perform  a variety  of 
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missions.  Wing  mounting  of  these  stores  can  cause  significant  reductions  in 
wing  flutter  speeds  and  can  give  rise  to  different  flutter  modes  witn 
significantly  different  frequencies.  Passive  means  to  accomodate  these 
situations  result  in  structural  modifications  or  the  imposition  of  speed 
placards.  These  passive  methods  generally  reduce  aircraft  performance.  Thus, 
active  control  of  flutter  is  a promising  alternative  especially  with  the 
development  of  highly  reliable  fly-by-wire  technology.  To  accomodate  the 
variety  of  possible  flutter  modes  involved  in  wlng/store  flutter,  an  adaptive 
capability  is  highly  desirable.  Thus,  there  is  an  affirmative  answer  to  the 
second  question. 

With  regard  to  the  first  question,  this  application  raises  two  issues. 

Issue  1:  The  synthesis  technology  must  provide  stabilization  of  an  unstable 
system  with  a rapid  speed  of  response.  This  requirement  arises  from  the 
possibility  that  a change  in  store  configuration  can  cause  the  aircraft  to 
suddenly  have  an  unstable  flexure  mode  corresponding  to  the  new  store  con- 
figuration. To  add  to  the  challenge,  it  is,  of  course,  possible  that  the 
adaptive  controller  was  actively  stabilizing  the  flutter  mode  corresponding  to 
the  configuration  before  the  change.  This  requires  the  adaptive  controller  to 
adapt  from  one  unstable  oscillatory  mode  to  another  rapidly  enough  to  prevent 
structural  damage  to  the  aircraft. 

Issue  2:  The  synthesis  technology  must  apply  to  Infinite  dimensional 
dynamics  which  are  crudely  approximated  by  finite  dimensional  linear  systems. 

This  issue  is  common  to  all  flexure  control  problems  but  especially  Important 
for  ones  involving  high  frequency  aero-elastic  dynamics.  The  structural  and 

aerodynamic  models  used  in  synthesis  are  increasingly  inaccurate  with  ln- 

. I 

creasing  frequency. 
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A second  possible  application  that  is  currently  being  contemplated  is  the 


adaptive  control  of  the  Space  Shuttle  Orbiter.  Here  the  motivation  is  that 


there  will  be  significant  differences  in  payloads  with  attendant  differences 


in  center  of  mass,  moments  of  inertia,  and  changes  in  flexure  characteristics 


A controller  tailored  to  each  payload  could  be  designed,  but  adaptive  control 


would  be  very  convenient.  The  demands  on  the  technology  for  this  application 


appear  to  be  much  less  severe  than  the  wing/store  flutter  application.  How 


ever,  since  the  motivation  for  adaptive  control  is  convenience,  the  monetary 


cost  of  designing  such  an  adaptive  control  must  be  low  to  be  of  significant 


benefit  over  tailored  designs 


ROBUST  CONTROL 


Probably  each  participant  has  his/her  own  definition  of  robust  control 


I believe  the  two  major  robustness  properties  are  performance  Invariance  and 


stability  invariance.  Performance  invariance  relates  to  disturbance  rejection 


and  command  following  and  generally  Implies  that  the  return  difference  matrix 


is  large.  Stability  invariance  generally  implies  that  the  return  difference 


matrix  is  attenuated  sufficiently  at  high  frequencies  to  provide  margins  con' 


sistent  with  expected  uncertainty  levels 


It  is  my  opinion  that  every  concept  of  robust  control  should  give  con' 


slderation  to  each  of  these  properties.  Furthermore,  adaptive  synthesis 


techniques  used  to  treat  parameter  uncertainty  and  provide  performance  la 


variance  should  not  Ignore  robustness  with  respect  to  unmodeled  dynamics  and 


nonlinearities 
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COMMENTS  ON  AIRCRAFT  CONTROL  PROBLEMS 

David  K.  Bowser 
Group  Leader 
Control  Analysis  Group 
Flight  Control  Division 
Air  Force  Flight  Dynamics  Laboratory 
Wrlght-Patterson  Air  Force  Base,  Ohio  45433 

The  Control  Analysis  Group  was  organized  within  the  AFFDL  a year  and  a 
half  ago  to  further  research  and  applications  in  advanced  control  analysis 
methods,  We  initiated  new  efforts  and  carried  on  several  existing  efforts  in 
this  area.  A quick  rundown  of  these  efforts  follow.  In  addition,  a statement 
is  given  relative  to  a real  world  problem  area  that  may  show  significant  payoff 
through  the  application  of  adaptive  control  concepts. 

EFFORTS : 

Recently  a contract  was  awarded  through  our  office  to  Dr.  Mehra  of 
Scientific  Systems,  Inc.  using  basic  research  funding  from  AFOSR.  This  effort 
is  entitled  Basic  Research  in  Digital  Stochastic  Model  Algorithmic  Control. 

Two  efforts  in  the  area  of  analysis  methods  for  digital  control  systems 
have  been  funded  using  basic  research  funding.  Both  efforts  were  awarded  to 
Dr.  Whitbeck  of  Systems  Technology,  Inc.  The  first  effort,  entitled  Analysis 
of  Digital  Flight  Control  Systems  with  Flying  Qualities  Application,  has 
resulted  in  AFFDL  TR-78-115.  The  second  effort  pursuing  direct  digital  design 
methods  was  recently  awarded.  It  is  entitled  Digital  Control  Systems  Synthesis 
Using  Multiple  Order  Sampling. 

The  Control  Analysis  Group  is  also  involved  with  Major  Gary  Reid  of  AFIT 
in  the  sponsorship  of  Masters  level  thesis  efforts  applying  Dr.  Mehra's  work  on 
Model  Algorithmic  Control  Application  to  B-52  flutter  mode  control  problems. 

In-house  work  is  also  being  accomplished  relative  to  the  formulations  of  a 
definitive  statement  of  Integrated  control  concepts  which  encompass  six-degree- 

i m\mm 


of-freedom  digital  flight  control,  fault  tolerant  design,  microprocessors,  and 
parallel  processing.  This  effort  is  largely  a planning  effort  at  this  stage, 
but  will  focus  and  harmonize  a significant  group  of  existing  and  planned 
efforts  related  to  integrated  control  within  our  division. 

PROBLEM  AREA  FOR  ADAPTIVE  CONTROL: 


The  near  stall  flight  regime  is  fraught  with  highly  nonlinear  changes  in 
dominant  aerodynamic  coefficients  such  as  Cng,  C^,  and  Cm^.  Linear  analysis 
and  synthesis  methods  are  normally  used  for  control  system  design  which 
optimize  system  dynamics  at  lower  angles  of  attack  where  the  system  model  be- 
haves in  a more  nearly  linear  fashion.  Normally  the  designer  then  evaluates 
what  he  has  at  higher  angles  of  attack  through  nonlinear  simulation.  If  real 
problems  are  apparent,  the  designer  either  limits  the  aircraft  to  lower  angles 
of  attack  through  placards  placed  upon  that  flight  regime,  or  he  may  develop  a 
limiting  type  of  flight  control  system  that  inhibits  the  aircraft  from 
entering  the  dangerous  flight  regimes  existing  at  higher  angles  of  attack. 

It  is  apparent  that  if  adaptive  control  systems  could  be  designed  to  follow 
the  vehicle  dynamics  in  the  near  stall  flight  regime,  and  if  sufficient  control 
power  could  be  generated  to  provide  the  necessary  level  of  control,  then  real 
improvements  in  safety  of  flight,  as  well  as  increased  usable  maneuverability, 
could  be  achieved.  Controlled  flight  at  high  angles  of  attack  is  a real  problem. 
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ON  ROBUSTNESS 
Michael  G.  Safonov 

Department  of  Electrical  Engineering— Systems 
University  of  Southern  California 
Los  Angeles,  California  90007 

The  state  of  the  art  In  adaptive  control  is  such  that  in  the  majority  of 
situations  truly  optimal  dual  control  solutions  are  computationally  infeasible. 
Consequently,  in  implementing  practical  adaptive  control  techniques  it  is 
almost  inevitable  that  one  must  make  simplifying  assumptions  or  approximations  — 
e.g.,  that  parameters  vary  slowly  or  that  the  system  operating  plant  changes 
slowly.  While  in  practical  situations  one  frequently  finds  that  these 
assumptions  and  approximations  are  quite  reasonable,  one  ultimately  must  resort 
to  simulation  or  to  stability  theoretic  techniques  to  establish  their  validity. 

Robustness  in  the  context  of  control  engineering  is  the  tolerance  of  a 
control  design  to  uncertainty  and  imprecision  in  modeling— including  impre- 
cision that  is  intentionally  introduced  in  the  form  of  simplifying  assumptions 
or  approximations.  The  significance  of  robustness  in  the  area  of  adaptive 
control  is  twofold.  First,  practical  adaptive  control  designs  must  have  a 
certain  degree  of  robustness  if  simplifying  assumptions  about  slowly  varying 
parameters,  operating  points  and  so  forth  are  to  be  valid.  Second,  robustness 
is  a significant  issue  in  adaptive  control  because  of  its  implications  about 
when  adaptive  behavior  is  even  necessary  in  a control  design:  if  one  can 
design  a control  law  under  the  extreme  simplifying  assumption  that  parameters 
and  operating  points  do  not  vary  at  all  and  if  that  control  law  is  sufficiently 
robust  then  adaptive  control  is  plainly  unnecessary. 

Efforts  to  quantitatively  characterize  robustness  have  inevitably  been 
related  to  stability  and  sensitivity  theory,  state-space  (Lyapunov)  and  input- 
output  techniques  both  being  effective.  However,  because  the  quantitative 
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amount  of  robustness  at  any  particular  node  in  an  interconnected  system  is 
directly  related  to  the  return-difference  at  the  node  (an  input-output  rela- 
tion), it  is  my  present  opinion  that  input-output  methods  provide  a more  direct 
and  conceptually  simple  method  to  address  robustness  issues  in  control,  the 
role  of  the  state-space  being  primarily  in  the  representation  of  input-output 
relations  for  computational  purposes.  My  recent  research  in  the  area  of 
robustness  reflects  this  view.  Robustness  of  the  property  of  stability  is 
addressed  in  an  input-output  setting  in  refs.  1-4,  6,  7.  Robustness  of  system 
response  (i.e.,  sensitivity)  is  addressed  in  ref.  5.  Reference  4 contains  a 
result  which  is  especially  relevant  to  adaptive  control:  one  can  substitute 
nondivergent  estimates  (e.g.,  estimates  from  a globally  "incrementally"  stable 
nonlinear  observer)  for  true  values  in  a control  system  without  inducing  in- 
stability. Nondivergence  is  a property  of  the  estimator  itself  and  is  not 
control  law  dependent.  So,  insofar  as  stability  is  concerned  the  design  of  an 
adaptive  controller  can  with  complete  rigor  be  separated  into  two  parts,  a 
parameter  estimator  and  a parameter-dependent  control  law. 
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NONROBUSTNESS  AND  BIFURCATION* 

Raman  K.  Mehra 
Scientific  Systems,  Inc. 

186  Alewife  Brook  Parkway 
Cambridge,  MA  02138 

Bifurcation  phenomena  in  nonlinear  systems  is  known  to  cause  extreme 
sensitivity  of  system  behavior  to  parameter  variations.  We  consider  the 
specific  case  of  Llnear-Quadratic-Gaussian  (LQG)  control.  The  nonrobustness 
of  LQG  is  related  to  the  bifurcation  behavior  of  the  Riccati  equation. 
Specifically,  the  null  solution  of  the  Kalman  filter  Riccati  equation  for  the 
zero  process  noise  case  bifurcates  when  an  eigenvalue  of  the  system  crosses  the 
imaginary  axis  from  left  half  to  right  half  plane  (transition  from  a stable  to 
an  unstable  system).  Another  bifurcation  of  the  Riccati  equation  occurs  for 
transitions  from  minimum  phase  to  nonminimum  phase  characteristics,  i.e.,  when 
one  of  the  zeros  of  the  system  crosses  the  imaginary  axis  from  the  left  to  the 
right  half  plane.  The  latter  case  occurs  when  the  measurement  noise  covariance 
is  zero.  The  above  two  bifurcations  of  the  Riccati  equation  explain  the 
difference  in  robustness  properties  of  the  LQ  design  versus  LQG  design.  The 

latter  loses  robustness  as  the  open  loop  system  dynamics  become  unstable  or 

j 

nonminimum  phase. 
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A NEW  FORMULATION  OF  THE  MULTIVARIABLE  ROBUST 
SERVOMECHANISM  PROBLEM 

G.  F.  Franklin 

Department  of  Electrical  Engineering 
Stanford  University 
Stanford,  California  94305 


The  design  of  multivariable  control  systems  to  provide  zero  steady-state 
system  error  in  the  presence  of  non-decaying  disturbances  and  non-decaying 


reference  signals  and  in  spite  of  perturbations  to  system  parameters  has  been 

12  3 

formulated  and  solved  by  Davison  in  a series  of  papers  ’ * and,  using  geo- 
metric methods,  by  Franc is, Wonham^*^  and  Pearson**.  The  latter  authors 
present  the  essential  structure  of  the  solution  as  the  "Internal  model 


principle".  The  version  of  the  problem  to  be  studied  here  may  be  described 


by  the  following  equations 


state 

x * Fx  + Gu  + GjW 

n xl 
s 

(1) 

output 

y * Hx  + Ju  + J^w 

n xl 

0 

(2) 

error 

e =>  y-r 

n xl 
o 

(3) 

reference 

t<p>  - \ a,r(p-‘> 
i-1  1 

; r(0) 

unknown  n xl 
o 

(4) 

disturbance 

(P)  * (P-i) 

wv  - 2 ’ 

i-1  L 

; w(0) 

unknown  n.xl 
a 

(5) 

control 

U - k(y,e) 

n xl 

P P-i 

In  (4)  and  (5),  the  are  real  scalars.  If  a(s)  * s -^1  , then  a(s) 

is  the  polynomial  of  lowest  degree  for  which  r and  w satisfy  the  corresponding 
differential  equation. 

The  problem  is  to  design  a control  law  k(y,e)  to  provide  regulation, 
which  is  to  say  that  the  error,  e,  tends  to  zero  as  time  gets  large  even 
(especially!)  if  the  are  such  that  r and  w grow  without  bound  with  time. 
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The  control  must  also  be  structurally  stable  or  robust  in  the  sense  that 
regulation  occurs  in  the  presence  of  perturbations  of  the  greatest  possible 
number  of  system  parameters. 


Part  I.  Regulation 

The  state  of  the  dynamical  system  described  by  (1),  (4),  and  (5)  is  of 
dimension  ng  + P • nQ  + P • nd»  Since  the  control  signal  does  not  appear 
in  (4)  or  (5)  it  is  clear  that  this  system  is  not  controllable.  However  the 
task  is  to  control  the  error,  e(t).  Since  the  error  is  a linear  function  of 
the  overall  state,  one  should  be  able  to  find  a coordinate  system  within  which 
the  error  is  directly  a coordinate  of  the  state.  The  problem  of  error  control 
may  then  be  expressed  in  the  well  known  terms  of  state  control. 

Since  the  error  contains  the  reference  input  and  that  signal  satisfies 
differential  equation  (4)  which  has  PC^  order  derivatives,  an  equation  in  the 
error  of  similar  structure  is  considered. 


.<P>  - l - y<P>  - l P(y<p-1> 

i-1  1 i-1  1 


- r<P>  + 2 ■ r^ 

i-1  1 


(6) 


Substituting  (4)  for  r^  in  (6) 


e<P)  - I B1.<p*1)  - y<P>  - \ Bty^P_1) 

i-1  1 i-1  1 


p 

+ Z (01-a.)r(p"i)  . (7) 

i-1  1 1 


It  is  noticed  that  the  (uncontrollable)  reference  input  will  vanish  from  the 
error  equation  if  and  only  if  - a^.  If  this  selection  is  made  and  y is 
expanded  from  (2),  (7)  becomes 


I 


■ 


■ I 

(I 

i 

i 


+ J[uvr'  - E 
t-1 


V 


+ J,[w 


(P) 


P 

- E 
i-1 


(8) 


The  reason  for  selecting  a(s)  to  describe  the  dynamics  of  both  r and  v becomes 
clear:  the  disturbance  vanishes  from  (8)  because  of  (5),  and  there  is  a 
possibility  of  controlling  the  error  state  from  the  Input  u.  To  complete  the 
description  the  plant  state  x is  replaced  by  § defined  as 


§ 


.(*> 


P 

E 

i-1 


c^x 


(P‘i) 


(9) 


and  Che  control  is  replaced  by 


then  (8)  becomes 


(P)  1.  (P-i) 

u * uv  E a.uv 

i-1  1 


e(P)  - E a,e<P-i)  - H§  + Jn  . 
i-1 


(10) 


(11) 


The  state  equation  for  § is  given  by 

5 - ,(P+l>  - Z a,x(p-1+l> 
i-1  l 


- Fx(?)  + Gu(p)  + 


»•*>  - E atGu'p-‘>  - I alG1»(p-l> 


E a,Fx 
i-1 


i-1  * i-1 

Since  the  are  scalars,  a^F  * Fa^  and  (12)  may  reduce  to 


(12) 


5 - F?  + G^ 


(13) 





Equations  (11),  (13),  and  (3)  now  describe  the  overall  system  state  and  the 


portion  containing  the  error  is  given  by  (11)  and  (13).  In  state  variable 


form  these  are 


where  Z 


The  error  can  be  forced  to  zero  if  and  only  if  (A,B)  are  stabilizable  and 


can  be  given  arbitrary  dynamics  if  (A.B)  are  controllable.  Only  the  coa 


ditions  for  controllability  are  presented 


The  matrices  (A,B)  are  controllable  if  and  only  if 


By  elementary  row  operations,  (16)  is  reduced  to  the  condition 


rank 


Z'  equals  the  transpose  of  Z 
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Since  Che  matrix  in  (17)  has  n + P • n rows,  all  Che  rows  must  be  independent 

so 

and  especially  Che  last  n rows  must  be  independent  for  all  s.  These  are  Che 

3 

rows  of  [si  - F:G]  and  requires  thac  (F,G),  Che  plant,  be  controllable.  If  we 
let  s « X^  when  a(X^)  » 0 (X^  is  a characteristic  value  of  the  dynamic  systems 
which  produce  r and  w)  it  is  obvious  that  the  final  condition  for  controll- 
ability  is  given  by 


rank 


j -H  J 

Xt-F  G 


n 


a 


+ n . 
o 


(18) 


Since  this  matrix  has  n + n columns,  and  the  rank  is  no  more  Chan  the  minimum 

s c ’ 

of  (n  + n ; n + n ),  it  is  required  that  n > n , or  that  there  must  be  as 
S C S O C *“  o 

many  controls  as  there  are  outputs.  A value  of  X^  for  which  the  rank  of  (18^ 

is  less  than  n + n is  a zero  (7,8)  of  the  plant.  In  summary,  the  error  state 
s o 

is  controllable  if  and  only  if 
(i)  F,G  is  controllable 

(li)  nc  > nQ  (19) 

(iii)  H,F,G,J  has  no  zeros  at  X^  for  which  a(X^)  ■ 0. 

If  the  conditions  of  (19)  are  met,  then  there  exists  a control  law 
^ ■ -Kz  so  that  the  error-state  system  of  (14)  has  an  arbitrary  characteristic 
equation.  The  control  gain  K may  be  computed  by  any  method  such  as  optimal 
quadratic  loss,  pole  assignment,  or  frequency  response  synthesis. 


Part  II.  Implementation 

Once  the  control  law  is  designed,  it  is  necessary  to  implement  it  by  con- 
structing the  plant  control  u,  from  the  system  error  e and  the  plant  state,  x, 
if  it  is  available,  or  from  an  estimate  of  the  state  if  only  the  output  is 
sensed.  For  this  development,  the  control  gain  is  first  partitioned  according 
to  Che  elements  of  the  state  z as 
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u - -K  e 
* P 


Kle 


(P-D 


• Ko5 


(20) 


If  (9)  and  (10)  are  substituted  into  (20)  for  § and  p,,  the  controller  equations 
become 


u<P)  - Z a1u(P"i)  « - Z K.e(P"i)  - K fx(P)  - Z a,x(P’i) }. (21) 
i-1  1 i-1  1 0 1-1  1 


Recognizing  that  u and  x enter  (21)  by  equations  with  identical  coefficients, 
(21)  can  be  written  as 


(u 


+ Kqx) 


(P)  . 


P 

Z 

i-1 


aA(u 


KoX) 


(p-i) 


p 

- z 

i-1 


Kie 


(P-I) 


(22) 


It  is  noticed  immediately  that  (22)  represents  the  servocompensator  of 
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Davison  ’ * and  the  internal  model  of  Francis  and  Wonham.  As  an  aside  from 

the  perspective  of  classical  control,  it  is  well  known  that  system  error  in  a 
unity  feedback  topology  is  reduced  by  the  loop  gain.  The  requirement  that  the 
error  be  zero  at  frequencies  X^  such  that  a(X^)  - 0 would  be  expected  to  call 
for  Infinite  gain,  i.e.,  poles,  at  X^. 

However,  the  control  of  (22)  assumes  that  all  the  plant  states  are  avail- 
able. If  only  the  error  e and  the  output  y are  measured,  it  is  possible  to 

9t 

estimate  x by  a linear  observer.  A suitable  (but  not  minimal  order) 
estimator  equation  is^ 

i - Fx  + Gu  - L(y  - Ju-Hx)  . (23) 

The  estimate  error  associated  with  (23)  is 


^One  may  also  control  (14)  by  a dynamic  compensator,  of  course.  ^ 

^The  estimator  in  (23)  can  use  any  measurement  of  the  state  for  correction 
in  place  of  the  system  output  y.  Some  authors  identify  ym  as  the  signal 
to  be  measured  to  distinguish  it  from  the  output  to  be  controlled. 
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^ • 

x-x-x-Fx  + Gu  + GjW  - Fx  - Gu 

- L(Hx  + JjW  - Hx) 
x - (F-LH)x  + (Gx  - LJ^w  . 


(24) 


If  F, H are  observable,  then  L may  be  selected  to  give  F-LH  an  arbitrary 
characteristic  equation.  However,  since  w is  a possibly  growing  signal,  it  is 
likely  that  x will  also  grow  in  time.  It  is  necessary  to  demonstrate  that  use 
of  x from  (23)  in  place  of  x in  the  controller  (21)  will  not  cause  the  system 
error,  e,  to  fail  to  tend  to  zero.  If  x is  replaced  by  x in  (21)  there  results 
the  modified  controller  equation 


u 


(P) 


P 

- Z 

i-1 


V 


(P-i) 


P 

- Z 
i-1 


V 


(P-i) 


P 

- Z a.*(P"i>)»(25) 

i-1  1 


Now  x - x-x  and  x^  - x^  - x^k*.  Therefore  (25)  is  equivalent  to 


,(P> 


1 (P-i) 

• Z a.u' 

i-1  1 


- £ - (K  ,<P>  - £ a,.(P-l)l 

i-1  1 0 i-1 


+ K {x(P)  - Z cl1X(P"1)  } • (26) 

° i-1 


If  (24)  is  used  to  evaluate  the  last  term  in  (26),  one  obtains 
-(P+l)  . £ a ?(P‘i+1)  . (F-LH)[x(P)  - Z a.x^"1*]  . 

j _ t * 1 


i-1 


i-1 


(27) 


Defining  ^(t)  - *(P)  - Z a.X(P"i),  (27)  is 

i-1  X 


H - (F-LH)T) 


(28) 


and  (26)  is  reduced  to 


122 


1 


! 


1 

! 


a 
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(P-i) 
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- 2 
i-1 


Kte 


(P-i) 


- 2 a,x(p-l) } + K Ti<t). 
i-1  1 0 


(29) 


Thus  the  effect  of  using  the  estimated  state  is  to  cause  the  control  to  have 
added  a term  KqT]  which  is  the  output  of  (28),  a dynamic  system  with  a character- 
istic equation  arbitrarily  selected  by  choice  of  L if  F,H  is  observable. 

Part  III.  Robustness 

From  the  nature  of  the  pole-assignment  problem,  the  state  Z in  (14)  will 
tend  to  zero  for  all  perturbations  in  the  system  parameters  which  leave  A-BK, 
and  F-LH  stable.  However,  if  the  elements  of  H,  J,  or  in  (2)  are  changed, 
Chen  the  system  will  force  the  perturbed  output  to  be  equal  to  the  reference 
signal.  Thus  the  system  forces  the  output  of  the  sensors  to  track  the 
reference  signal  and  is  not  robust  with  respect  to  perturbations  in  the  sensor 
parameters.  The  second  limitation  on  robustness  derives  from  (7)  where  it  was 
required  that  8^  * if  the  reference  and  disturbance  signals  are  to  vanish 
from  the  error  state.  The  solution  is  thus  not  robust  with  respect  to  the 
selection  of  the  in  the  control  law  (22).  From  the  point  of  view  of  the 
implementation,  one  can  say  that  the  values  of  implemented  define  the  class 
of  reference  and  disturbance  signals  for  which  the  error  certainly  goes  to  zero. 
If  the  system  is  subjected  to  signals  which  fail  to  satisfy  (4)  or  (5)  with 
as  implemented  in  (22),  then  the  error  cannot  be  guaranteed  to  tend  to  zero. 

An  important  special  case  with  roots  deep  in  classical  control  occurs  when  all 
— 0.  The  signals  defined  by  (4)  and  (5)  are  then  polynomials  in  time  and 
the  controller  can  typically  be  implemented  as  a chain  of  integrators  with  great 
accuracy. 


Example  1 illustrates  the  introduction  of  integral  control  to  a first 


order  plant 


A control  gain  which  gives  the  characteristic  polynomial  (s+1)  +4  is 


Equation  (30)  shows  explicitly  the  integral  control  on  the  system  error 


Example  2.  A more  complex  example,  but  still  single  input  single  output 


is  motivated  by  a servomechanism  to  follow  the  data  track  on  a computer  disk 


memory  system.  Because  the  data  track  is  not  exactly  a centered  circle,  the 


radial  servo  must  follow  a ("runout")  sinusoid  input  of  radian  frequency  u> 


The  (normalized)  parameters  are 


at- 


124 


• I .*] ; c ■ [!] ; 

' o' 

J - 0 ; J.  - 0 ; G,  - 

L°. 


H - [1  O]  ; 


r ■ -u>  r . 


The  error  state  matrix  is 


5 B 


The  characteristic  equation  of  A-BK  Is 


s4  + KQ2s3  + (u>2+K01)s2  + (K1-Hb2K02)s  + K2  + U)2KQ1  - 0 


from  which  the  gain  may  be  selected  If  pole  assignment  Is  satisfactory  for  the 
design.  Since  x^  is  the  output  of  the  plant,  it  is  available  for  feedback. 

An  estimator  for  x2  could  be  described  by  the  equation  (x^  - is  used  as 
the  "measurement"  of  x2  for  the  estimator  design): 


x2  - -x  + u + L(x^  - x2)  . 


The  estimator  error  equation  is 


x2  - -(+1+L)x2 


The  estimator  gain  may  be  selected  from  (33).  The  "servocompensator"  con- 
sisting of  the  oscillator  with  frequency  u>  is  clearly  seen. 
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Conclusions:  The  robust  servomechanism  problem  has  been  formulated  in  the 
error  space  which  allows  an  alternate  derivation  of  most  of  the  known  results 
using  only  the  theory  of  controllability.  It  is  believed  that  this  formulation 
is  in  some  ways  simpler  than  previous  presentations. 
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A UNIFICATION  OF  ADAPTIVE  AND  ROBUST  CONTROL  CONCEPTS* 

K.  D.  Young 

Department  of  Mechanical  Engineering  and  Mechanics 
Drexel  University 
Philadelphia,  Penn.  19104 

The  naivety  of  the  idea  of  changing  the  controller  as  plant  parameters 
vary  from  which  the  adaptive  control  concepts  are  derived  is  soley  responsible 
for  the  multitude  of  adaptive  control  structures  of  various  degrees  of  sophis- 
tication ranging  from  the  self-optimizing  types  of  the  early  days  to  the  self- 
tuning type  and  the  model  reference  adaptive  type  of  today.  The  design  goal 
of  these  adaptive  control  schemes  is  to  assure  that  plant  performance  specifi- 
cations (such  as  stability)  are  met  under  all  foreseeable  plant  parameter 
variations  often  caused  by  changes  in  operating  conditions.  This  same  design 
goal  is  shared  by  control  engineers  whose  concern  is  essentially  that  if  a 
fixed  gain  feedback  controller  is  designed  for  the  nominal  plant  whether  it 
remains  effective  as  the  plant  parameter  changes.  The  robust  control  concept 
is  the  outcome  of  this  concern:  design  fixed  gain  feedback  controller  that  are 
robust  in  the  sense  that  plant  performances  remain  satisfactory  when  the  plant 
parameters  vary  arbitrarily  within  a certain  set. 

Robust  control  and  adaptive  control  concepts  are  often  considered  to  be 
distinctively  different.  It  is  customary  to  associate  imnediately  linear  fixed 
gain  feedback  controller  structures  with  robust  control  concepts  and  complex 
nonlinear  controller  structures  with  adaptive  control  concepts.  The  distinction 
between  robust  control  and  adaptive  control  concepts  becomes  superficial  when 
recently  a model  reference  adaptive  control  problem  is  solved  using  the  theory 
of  variable  structure  systems  and  sliding  mode.1.  The  resulting  adaptive 

This  work  is  partially  supported  by  the  Office  of  Naval  Research  under 
contract  N00014-77-C-0642. 
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2 

controller  is  a variable  structure  feedback  controller  and  indeed  possesses 
an  inherent  adaptation  mechanism.  The  success  of  this  adaptive  control 
solution  can  be  accredited  to  the  achievement  of  parameter  insensitivity  when 
this  adaptive  control  system  is  in  sliding  mode.  In  the  robust  control  frame- 
work, variable  structure  feedback  has  also  been  shown  to  be  effective  in  pre- 
serving plant  performances  under  plant  parameter  variations  because  of  the 
inherent  insensitivity  properties  of  variable  structure  feedback  systems. 
Variable  structure  control  in  fact  can  be  viewed  as  either  a robust  control  or 
adaptive  control  concept.  In  the  robust  control  context,  it  is  known  that 
variable  structure  feedback  controller  has  the  same  insensitivity  property  of 
high  fixed  gain  linear  feedback  controller.  In  the  adaptive  control  context, 
it  is  indeed  a controller  with  changing  parameters  and  the  controller  structure 
is  varied  in  a true  adaptive  sense. 


^K.  K.  Young,  "Design  of  Variable  Structure  Model  - Following  Control  Systems," 
IEEE  Trans.  Automatic  Control.  Vol.  AC-23,  pp.  1079-1085,  1978. 

2 

A variable  structure  feedback  controller  in  this  case  is  a linear  feedback 
controller  whose  feedback  elements  are  discontinuous  on  some  switching  hyper- 
planes. 
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SOME  ASPECTS  OF  INSENSITIVE  AND  ADAPTIVE  CONTROL 

Gerhard  Kreisselmeier 
Institute  for  Dynamics  of  Flight  Systems 
Oberpfaffenhofen,  D-8031 
Wessling,  F.R.  Germany 

1.  ON  INSENSITIVE  CONTROL  VERSUS  MRAC 

It  is  well  known,  that  feedback  can  reduce  the  effect  of  plant  parameter 
i variations.  See,  for  example,  the  conditional  feedback  structure  of  Fig.  1. 

M The  error  signal  Ay  Is  fed  back  to  make  the  transfer  behaviour 

J {uc  -*  y}  ■ £u  - yN)  for  all  operating  conditions. 

|r| 

Fig.  1 

Comparison  sensitivity  re- 
duction control  structure. 


129 


(2)  The  disturbance  rejection  behaviour  as  well  as  the  disturbance 
sensitivity  depend  on  the  same  feedback  and  must  be  taken  into 
account  in  this  compromise. 

(3)  The  range  of  parameter  variations,  which  can  be  covered  also  depends 
on  the  feedback  gains  and,  due  to  (1),  may  be  not  very  large. 

(4)  Linearity  of  both  the  plant  and  the  feedback  law  substantially 
simplify  the  design  and  allow  easy  closed  loop  system  analysis. 

It  is  the  linearity  (4)  which  makes  insensitive  control  particularly 
attractive  for  practical  applications.  Here  a lot  of  experience*,  is  available 
and  no  such  basic  problems  comparable  to  the  stability  problem  in  adaptive 
control  exist. 

It  is  the  limited  parameter  range  (3),  which  is  the  drawback  of  insensitive 
control.  However,  using  a reformulated  sensitivity  concept,  which  removes  the 
low  sensitivity  - high  loop  gain  interrelation,  very  large  parameter  ranges 
can  be  covered  as  will  be  shown  in  Section  11a. 

As  a natural  extension  of  the  control  structure  of  Fig.  1 we  have  the 
typical  MRAC  structure  of  Fig.  2. 


Fig.  2 

Model  reference  adaptive  control 
structure 
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Again  the  control  law  is  to  make  the  transfer  behaviour  [u£  -*  y}  ■ 
fu  -»  y„]t  But  now  the  control  is  modified  to  be  nonlinear,  where  the  non- 
linearity  is  the  outcome  of  the  idea  to  adapt  the  control  in  order  that 
{uc  "*  y}  matches  [u^  -•  y^}  asymptotically. 

The  main  advantages  of  MRAC  over  insensitive  control  are: 

(5)  Zero  sensitivity  is  obtained  asymptotically  where  only 

(6)  finite  feedback  gains  are  involved. 

(7)  Arbitrarily  large  parameter  ranges  can  be  covered  with  asymptotic 
zero  sensitivity. 

These  advantages  are  associated  with  the  following  substantial  short* 
comings : 

(8)  The  control  system  is  highly  nonlinear,  which  c omplicates  the  design 
and,  in  particular,  the  closed  loop  system  analysis. 

(9)  To  prove  global  stability  requires  a minimum  phase  plant  as  well  as 
a priori  knowledge  such  as  the  relative  degree  of  the  plant  transfer 
function  or  the  sign  of  its  static  gain. 

(10)  The  disturbance  re  lection  behaviour  is  essentially  unresolved  both 
from  the  (deterministic)  stability  and  the  regulation  accuracy  point 
of  view. 

Nevertheless  MRAC  does  work  in  practice  and  major  progress  in  proving 
global  stability  of  such  schemes  is  being  made  right  now.  Obviously  a MRAC 
design  is  subject  to  practical  constraints  imposed  on  the  adaptive  gains  and 
hence  the  speed  of  adaptation  as  well  as  to  the  disturbance  rejection  behaviour, 
similar  to  the  insensitive  design  (1)  - (2). 


At  this  point  a very  strange  facet  of  adaptive  control  is  worth  mentioning, 
to  illustrate  the  basic  stability  problems: 
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(11)  In  adaptively  controlling  a stable  plant  serious  stability  problems 
arise,  which  have  not  or  only  partially  been  solved  so  far.  This 
is  surprising  because  any  Instability  in  this  case  can  only  be 
generated  by  the  controller  itself,  l.e.  the  controller  is  too 
"stupid"  to  recognize  his  own  destabilizing  action. 

Some  of  the  stability  problems  in  MRAC  may  be  due  to  its  "zero  sensltl* 
vity"  concept: 

(12)  It  is  well  known  from  insensitive  systems,  that  (stable)  zero 
sensitivity  (by  infinite  gain)  can  be  accomplished  if  and  only  if 
the  plant  is  minimum  phase.  The  same  reasons  may  be  why  MRAC 
stability  has  been  shown  so  far  only  for  such  plants. 

(13)  Exact  model  matching  is  a mathematically  tractable  concept  (having 
its  own  limitations  and  shortcomings)  rather  than  an  actual 
practical  need. 

Furthermore,  it  may  be  physically  unfeasible  that  the  plant  behaves 
exactly  as  a fixed  model  over  a wide  range  of  operating  conditions.  The  latter 
will  be  substantiated  by  an  insensitive  control  example  in  the  subsequent 
section. 

II.  SOME  ADVANCES  IN  ADAPTIVE  CONTROL 

a)  Insensitive  control  for  very  large  parameter  ranges  ; 

The  physical  background  and  the  modified  sensitivity  concept  are  best 
introduced  by  means  of  an 

H 

Example:  pitch  rate  control  of  a McDonell  Douglas  F-4C  aircraft 
(longitudinal  motion  stability  augmentation  system). 

Figure  3 (left  curves)  show  the  uncontrolled  response  of  the  pitch  rate  q 

due  to  a step  elevator  deflection  for  five  extremal  flight  situations.  It  is 

obvious,  that  it  would  be  physically  unfeasible  to  make  the  response  in  the 

I v; 

1 
' 


HI 
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low  speed  landing  approach  (Curve  1)  as  fast  as  It  Is  possible  In  a high  speed 
situation  (Curves  2,  4).  Therefore,  maintaining  the  same  model  response 
q (t)  for  all  flight  conditions  cannot  be  the  design  goal  neither  for  an  in* 
sensitive  nor  an  adaptive  control  design. 

Instead,  we  use  the  following 


goal  for  insensitive  design: 


lq(t)  Z qM(at  t) 

chosen  individually  for  each 
flight  condition  1 


i.e.  we  require  the  response  to  the  basically  the  same  apart  from  the  time 
scales  at.  Adequate  choices  of  allow  the  aircraft  to  behave  more  slowly 
in  the  low  speed  situation  and  faster  in  a high  speed  one.  This  is  in  accor- 
dance both  with  the  plant  physics  and  the  usual  flight  control  specifications. 

Using  this  sensitivity  concept  let  the  transfer  behaviour  of  the  con- 
trolled plant  be  {u  -q.},  {u  — q } for  flight  conditions  i and  J respec- 
c l c j 

tively.  Then,  assuming  linear  feedback  the  transition  from  case  i to  J results 
in  a change  of  the  plant  input  u by  » k(Aq^). 

Zero  sensitivity  now  would  imply 


AutJ  4 °»  Aflij  - qM,i  - q^ j 4 0 ■>  "finite  loop  gain"  , 


i.e.  if  zero  sensitivity  in  the  above  sense  could  in  fact  be  accomplished  by  a 
linear  control  structure,  then  this  does  not  require  infinite  feedback  gains, 
different  from  the  usual  comparison  sensitivity  design  discussed  in  Section  l. 

The  results  of  a detail  design  are  depicted  in  Figs.  3-6.  Figure  3 shows 
the  transfer  behaviour  being  well  controlled  for  all  flight  conditions. 

Figure  4 shows  that  the  same  is  true  when  a step  disturbance  is  added  to  the 
system  input. 


Fig.  5.  Structure  of  the  insensitive  controller.  (k^  c^,  are 


the  same  for  all  flight  conditions;  q ■ pitch  rate  [o/a], 
T|  ■ elevator  deflection  [o]). 


t 


Fig.  6.  Bode  plot  of  the  feedback  gain  G(s)  ■ q(s)/T)(s) 
(q[o/s],  T][o];  G(Ju>)  has  no  phase  advance!). 
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It  Is  Important  to  note,  that  the  controller  structure,  which  is  shown  in 
Fig.  5,  is  extremely  simple  and  does  not  contain  any  of  the  above  models.  The 
latter  only  serve  as  a means  for  determining  suitable  controller  coefficients. 
Finally,  Fig.  6 shows,  that  only  low  feedback  gains  are  involved.  Further 
design  details  are  contained  in  [1]. 

In  conclusion,  large  parameter  ranges  can  well  be  covered  by  insensitive 
control.  Furthermore,  the  physics  of  the  plant  must  be  taken  into  account  also 
in  MRAC  (for  example,  by  adapting  the  model)  if  large  parameter  ranges  are  to 
be  covered.  This  may  require  a certain  amount  of  a priori  knowledge  of  Che 
plant  even  for  adaptive  systems. 


b)  An  adaptive  observer  satisfying  a separation  property 

Although  stable  adaptive  observers  for  unknown  linear  systems  have  been 
available  since  1972,  no  results  are  known  so  far  about  the  stability  of  a state 
feedback  control  system,  where  an  adaptive  state  estimate  instead  of  the  true 
state  is  fed  back.  Such  a control  structure  is  shown  in  Fig.  7. 


Fiq.  7 

Closed  loop  system  with 
an  adaptive  observer. 


The  main  reasons  that  the  (nonlinear)  closed  loop  stability  problem  has 
not  been  solved  are: 

(14)  The  so  far  available  adaptive  observers  required  uniform  boundedness 
of  u,  y to  prove  convergence  of  x - x. 
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(15)  but  to  prove  stability  of  the  closed  control  loop  uniform  bounded- 
ness of  u.  y cannot  be  assumed  but  must  be  proven. 

In  [2]  a new  adaptive  observer  has  been  suggested  which  removes  the 
uniform  boundedness  assumption  and  obtains  convergence  of  it  -x  even  in  case  of 
u,  y increasing  unboundly  as  t -*  «o.  This  is  essentially  achieved  by 

(16)  additional  observation  of  exp(F  t)[xQ  - xq]  where  F contains  the 
observer  dynamics 

(17)  additional  control  of  the  adaptive  gains  (and  hence  the  speed  of 

the  adpative  process)  according  to  the  signal  level. 

For  this  adaptive  observer  the  following  is  shown  in  [2]. 

T 

Theorem;  If  (A  + bk  ) is  a stability  matrix  and  the  adaptive  observer 
is  designed  to  be  asymptotically  stable,  then  the  control 
structure  of  Fig.  7 is  also  asymptotically  stable. 

In  particular  no  assumptions  on  the  system  parameters,  the  observer 
eigenvalues  and  the  speed  of  the  adaptation  are  made. 

A preliminary  copy  of  [2]  is  available  from  the  author. 

c)  Adaptive  control  via  adaptive  observation  and  on  line  adaptive  control 
law  synthesis 

The  work  of  [2],  briefly  outlined  in  Section  lib)  has  been  completed  in 
[3]  by  an  adaptive  law,  which  adapts  the  feedback  matrix  k from  the  Instantaneous 
system  parameter  estimates  generated  by  the  adaptive  observer.  The  adaptive 
control  structure  is  indicated  in  Fig.  8. 

In  [3]  a strategy  for  the  adaptive  control  law  synthesis  is  given,  which 
is  shown  to  result  in  global  asymptotical  stability  of  the  control  structure 
of  Fig.  8.  As  an  example,  the  so  far  unsolved  adaptive  pole  placement  problem 
has  been  solved  in  [3]. 


■ 


I 

] 
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Fig.  8 

Adaptive  control  via  adaptive 
observation  and  adaptive  control 
law  synthesis 


These  results  provide  substantial  theoretical  justification  for  various 
practical  applications,  where  similar  control  structures  have  been  used  success- 
fully. 

III.  ON  FUTURE  RESEARCH 

Adaptive  control  approaches  may  be  classified  into  those,  who  make  use  of 
a priori  knowledge  of  the  possible  plant  parameter  variations  and  those,  who 
do  not. 

Whenever  large  parameter  variations  occur  in  practice  they  will  be  well 
detectable  and  their  main  effects  will  be  identified.  So,  very  often  a priori 
knowledge  is  available  in  practice.  Note  that  such  a knowledge  is  also 
necessary  whenever  the  control  system  is  to  be  simulated  on  a computer  prior 
to  Implementing  the  adaptive  controller  in  practice.  The  latter  should  be  the 
rule  rather  than  the  exception. 

The  Insensitive  control  example  of  section  11a)  shows  how  necessary  this 
knowledge  may  be,  in  order  that  physically  unfeasible  design  goals  are  avoided. 
However,  most  of  the  adaptive  schemes  do  not  make  use  of  such  a knowledge  and 
much  research  should  be  made  how  to  incorporate  It  into  the  design.  As  a con- 
sequence, we  have  the 


J 
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first  prototype  design  situation:  Plant  and  all  possible  parameter 

variations  known. 


In  this  situation  a control  design  can  be  simulated  in  detail,  can  be 


optimized  and  in  particular  can  be  compared  with  other  competing  designs 


Accordingly  we  have  as  the 


second  prototype  design  situation:  Nothing  known  about  possible  para 

meter  variations. 


In  this  situation  arbitrary  parameter  variations  may  occur  and  therefore  the 


adaptive  law  must  be  as  skilled  as  a qualified  design  engineer  firstly  to 


maintain  physical  feasibility  of  the  control  action  and  secondly  to  guarantee 


good  regulation  accuracy.  It  is  obvious,  that  this  case  is  much  more  difficult 


theoretically  and  much  more  risky  in  its  practical  implementation  at  present 


Much  more  will  be  necessary  to  be  known  about  the  stability  properties  of 


adaptive  systems,  in  particular  when  unknown,  deterministic  disturbances  act 


upon  the  system.  Here  deterministic  stability  results  would  be  of  substantial 


value,  since  they  also  may  be  a precursor  to  general  results  on  the  disturbance 


regulation  accuracy.  Without  a well  established  theoretical  stability  back 


ground  adaptive  control  might  be  faced  with  a certain  reservation  from  potential 


applicants.  Also  the  failure  probability  would  be  difficult  to  analyze 


Finally,  we  have  seen  that  insensitive  control  can  cover  large  parameter 


variations.  Therefore  it  competes  with  adaptive  control,  at  least  as  far  as 


the  first  prototype  design  situation  is  concerned.  Further  developments  of 


such  insensitive  designs  and  analysis  of  the  feasible  parameter  ranges  would  be 


of  much  value.  One  could  also  develop  insensitive  controllers  with  nonlinear 


structures,  l.e.  combine  insensitive  and  adaptive  control  in  such  a way  that 


suitable  nonlinear  controller  structures  are  chosen  using  adaptive  ideas,  where 


as  the  design  of  the  free  parameters  in  such  structures  is  based  on  sensitivity 
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ADAPTIVE  CONTROL  SYSTEMS,  CLASSIFICATION,  PROBLEMS,  AND  SUGGESTIONS* 


H.  Kaufman 

Dept,  of  Electrical  & Systems  Engineering 
Rensselaer  Polytechnic  Institute 
Troy,  New  York  12181 


1.  BACKGROUND 

Implementation  of  control  systems  using  digital  logic  is  of  considerable 
interest  because  of  the  present  capabilities  for  designing  small,  lightweight 
and  inexpensive  minicomputers  and  microcomputers.  A feature  of  digital  logic 
which  makes  it  especially  advantageous  is  the  capability  for  implementation  of 
complex  control  systems  which  incorporate  high  order  nonlinearities  and 
multiple  loop  operations.  One  such  complex  control  structure  is  an  adaptive 
system  which  is  capable  of  on-line  adjustment  of  the  control  parameters  in 
response  to  changing  plant  characteristics. 

Two  distinct  types  of  adaptive  control  logic  can  be  considered: 

• Explicit  adaptive  controllers  in  which  on-line  estimates  of 
the  plant  parameters  are  used  for  gain  adjustment. 

• Implicit  adaptive  controllers  in  which  some  measure  of  the 
error  between  actual  and  desired  states  is  used  for  gain 
adjustment,  i.e.,  no  explicit  parameter  identification  is  used. 

Previous  studies  have  indicated  the  advantages  of  explicit  adaptation  if 
gain  magnitudes  are  constrained  and  if  large  parameter  variations  are  to  be 
expected.  However,  because  of  the  need  to  incorporate  an  on-line  identifier 
in  such  systems,  the  following  issues  arise: 

• Storage  and  timing  problems  in  the  implementation  of  on-line 
identification. 

if 
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• How  to  determine  which  parameters  need  to  be  identified. 

• Simultaneous  estimation  of  both  states  and  parameters  in  a 
noisy  environment. 

• Inability  to  guarantee  stability  of  the  controlled  system  in 
the  presence  of  imperfect  estimates. 

• Inability  to  determine  convergence  of  the  identifier  when  the 
control  signal  itself  depends  upon  the  identified  parameters. 

One  procedure  for  alleviating  these  problems  is  to  use  an  implicit 
adaptation  algorithm  which  takes  into  account  system  stability  requirements. 
However,  such  a design  requires  a means  for  rapidly  assessing  performance, 
which  in  turn  can  be  used  for  control  gain  adjustment.  Many  investigators  have 
made  use  of  comparisons  between  actual  plant  state  trajectories  and  those  from 
a model  chosen  to  reflect  desirable  operation  specifications.  Some  measure  of 
the  error  between  the  plant  and  model  state  vector  is  then  directly  used  for 
control  gain  adjustment. 

In  order  to  design  a controller  which  does  not  need  explicit  parameter 
estimates,  procedures  based  upon  Lyapunov  stability  and  hyperstability 
principles  have  been  applied  to  the  equations  defining  the  error  between  plant 
and  model. 

However,  in  order  to  prove  asymptotic  stability  these  designs  required  the 
satisfaction  of  certain  structural  or  matching  conditions  between  plant  and 
model,  sometimes  known  as  the  conditions  for  perfect  model  following  (PMF). 
These  conditions  are  in  general  not  satisfied  by  most  practical  plant  and  model 
systems.  If,  for  example,  the  model  is  chosen  to  reflect  desirable  decoupling 
characteristics  not  inherent  to  the  process,  then  the  PMF  conditions  most 
likely  will  not  be  satisfied.  Also,  if  the  model  is  constant  and  the  process 
is  time  varying,  then  it  is  very  likely  that  the  PMF  condition  will,  at  most, 


- 
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be  valid  only  for  short  intervals  of  time  (e.g.,  at  certain  nominal  conditions). 

To  date,  limited  results  have  been  developed  for  designing  implicit 
adaptive  controllers  in  the  absence  of  the  PMF  constraints.  At  best,  it  has 
been  shown  that  a controller  can  be  designed  for  a multivariable  continuous 
linear  system  which  stabilizes  (in  the  sense  of  boundedness)  the  error  between 
the  plant  and  model  state. 

2.  SUGGESTED  AREAS  OF  RESEARCH 

2.1  Explicit  Adaptive  Control  System  Design 

Because  explicit  adaptive  systems  can  potentially  encompass  a broad  range 
of  performance  indices  containing  penalties  on  the  control  gain,  it  is 
suggested  chat  research  and  developmental  efforts  be  geared  towards  the  design 
of  such  systems  for  future  computer  systems  which  might  alleviate  the  existing 
storage  and  timing  problems.  In  particular,  the  following  activities  are 
recommended: 

• Consider  the  use  of  parallel  architecture  for  rapid  on-line 
control  and  estimation. 

• Consider  the  use  of  indices  other  than  quadratics. 

• Consider  the  design  of  adaptive  controllers  for  nonlinear  systems. 

. Implement  adaptive  control  on  a physical  process. 

2.2  Implicit  Adaptive  Control  System  Design 

Because  of  the  attractiveness  of  an  implicit  adaptive  control  design,  it 
is  suggested  that  research  be  conducted  to  increase  the  applicability  of  the 
concept.  In  particular,  the  following  activities  are  recommended: 

• Design  implicit  adaptive  controller  suitable  for  digital 
multivariable  systems. 

• Consider  an  adaptive  controller  for  output  following. 


Consider  the  effects  of  noisy  state  and/or  output  measurements 


Design  implicit  adaptive  algorithms  for  nonlinear  systems 


Demonstrate  implicit  adaptive  control  of  a physical  system 


3.  CONCLUSION 


In  conclusion,  it  is  conjectured  that  many  of  the  problems  previously 


encountered  in  designing  explicit  adaptive  controllers  will  be  accomnodated 


using  computers  employing  very  large  scale  integration.  However,  although 
implicit  adaptive  controllers  are  very  attractive  because  of  their  relatively 


simple  structure,  more  analysis  is  required  to  extend  the  range  of  applicability 


145 


r 

[ 


j 


A 


A ■ »V.' 


COMMENTS  ON  ADAPTIVE  CONTROL 

E.  G.  Rynaski 
Staff  Engineer 
Advanced  Technology  Center 
Calspan  Corporation 
P.  0.  Box  400 
Buffalo,  New  York  14225 

Background 

During  the  late  1950's  and  early  1960's,  a lot  of  effort  was  put  into 
Adaptive  Control  research,  both  theoretical  and  experimental  research.  Some 
of  the  best  known  systems  were  developed  by  MIT,  Honeywell  and  Autonetics,  and 
several  adaptive  systems  were  actually  installed  and  flight  tested.  The  best 
known  flight  program  was  the  Honeywell  system  in  the  X-15  aircraft.  Less  well 
known  applications  include  the  F-lll  and  certain  missiles. 

The  general  idea  was  that  an  adaptive  system  should  maintain  constant 
dynamic  behavior  for  an  aircraft  over  an  extremely  wide  flight  range  during 
which  the  parameters  of  the  equations  of  motion  were  unknown  and  varying 
"rapidly."  As  it  turned  out,  however,  adaptive  control  was  overpromoted  and 
oversold.  With  one  possible  exception,  these  systems  did  not  work  as  adver- 
tised. 

A successful  application  required,  generally,  more  knowledge  of  the 
dynamics  of  the  airframe  than  was  previously  required,  not  less  knowledge,  and 
such  things  as  nonlinearities,  turbulence  and  structural  dynamics  made  success- 
ful operation  difficult,  if  not  impossible.  It  was  also  discovered  at  that 
time  that  invariant  dynamic  behavior  for  an  airplane  over  its  entire  flight 
range  was  neither  necessary  nor  particularly  desirable. 

As  a result,  adaptive  control  fell  into  disfavor  in  the  flight  community 
and  to  this  day  a residual  bias  still  exists. 


Recent  Activity 

The  recent  resurgence  of  Adaptive  Control  research  is  not  necessarily 
directed,  in  my  opinion,  towards  the  solutions  of  those  problems  that  caused 
the  downfall  of  the  original  adaptive  control  activity  and  the  same  pattern  of 
decline  can  be  repeated  if  we  are  not  careful.  In  addition  to  not  addressing 
the  old  problem,  consider  some  of  the  new  problems  associated  with  modern 
control  theory: 

1.  Quadratic  performance  Indices  for  control  criteria  specification. 

No  one  has  been  able  to  completely  successfully  relate  performance 
indices  to  flying  qualities.  Flying  qualities  specifications  are 
given  in  terms  of  poles  and  zeros  of  particular  transfer  functions  of 
an  airplane.  A relationship  exists  between  the  closed- loop  poles  and 
the  weighting  parameters  of  a performance  index  --  but  no  relationship 
has  yet  been  found  between  closed-loop  transfer  function  zeros  and 
performance  indices.  Non-minimum  phase  closed- loop  transfer  functions, 
often  generated  by  optimal  control,  are  generally  to  be  avoided.  How 


do  I choose  a performance  index  that  will  guarantee  minimum  phase 
closed- loop  transfer  functions? 

--  Failure  tolerance.  How  do  I choose  a performance  index  constrained 
such  that  stability  is  still  guaranteed  if  a sensor  failure  occurs? 
Will  there  be  time  to  switch  to  different  control  configurations? 
Should  observers  be  used  to  replace  the  lost  actual  measurement  or 
just  output  feedback? 

2.  Identification 

On-line  identification  is  not  yet  a household  word.  Our  experience 
with  off-line  identification  has  been  that  accuracy  is  a maybe  kind 
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of  thing,  very  much  dependent  upon  the  particular  airframe,  the 
quality  and  complement  of  instrumentation  on  the  aircraft,  the  feed* 
back  control  law,  the  model  form  and  the  input  design.  The  particular 
identification  algorithm  used  is  less  a factor  than  the  realities  of 
the  plant  itself.  The  same  flight  data  can  be  analyzed  for  weeks, 
adjusting  such  things  as  noise  covariances  and  instrument  biases,  and 
increasingly  more  accurate  results  are  obtainable.  Occasionally, 
after  the  first  successful  identification  has  been  laboriously  ob- 
tained, subsequent  identifications  can  be  performed  in  one  pass,  but 
only  within  a limited  flight  range. 

Consider  some  of  the  requirements  for  accurate  identification. 

1.  The  excitation  to  the  system  must  be  such  to  render  each  of  the  para- 
meters  Identifiable.  Can  successful  identification  be  accomplished 
using  normal  pilot  inputs  or  actual  turbulence  excitation?  Only  a 
flight  test  program  can  answer  this  question. 

2.  State  estimation  should  be  accomplished  independently  of  aerodynamic 
parameter  identification,  particularly  if  the  aerodynamics  are  non- 
linear. This  is  to  verify  the  consistency  of  the  instrumentation 
complement.  Instrumentation  scale  factors  and  bias  errors  change 
with  such  things  as  angle  of  attack;  gyros  drift,  the  c.g.  of  the 

* I i 

aircraft  changes,  etc. 

3.  The  model  form  changes.  An  aircraft  that  should  be  considered  4th  i 

order  at  slow  speed  may  be  considered  a 2nd  order  system  at  inter- 
mediate speeds,  but  must  be  treated  as  a nonlinear  system  at  transonic 


Mach  numbers  or  at  high  angle  of  attack.  Do  we  do  on-line  hypothesis 
testing  to  determine  the  correct  model  form? 
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4.  We  have  no  Idea  how  accurate  the  parameter  estimates  must  be  to 

either  guarantee  stability  or  to  insure  good  flying  qualities.  How 
accurate  is  accurate  enough? 

Very  important  will  be  the  cycle  time  for  a complete  computation; 
estimation  and  identification  or  hypothesis  testing  and  control  law  adjustment. 
For  most  manned  aircraft,  our  experience  has  been  that  a transport  lag  between 
the  time  that  a pilot  puts  a command  into  the  airplane  until  the  time  that  the 
airplane  starts  responding  to  that  command  must  not  exceed  40-60  milliseconds 
and  is  a function  of  the  vehicle  stability.  In  generaly,  any  delay  is  un- 
desirable. 

Need  for  Experimental  Research 

Many  of  the  questionable  aspects  of  adaptive  control,  particularly  with 
respect  to  manned  aircraft,  can  only  be  answered  by  an  experimental  research 
effort  that  parallels  the  theoretical  work.  Adaptive  control  suffers  from  a 

I 

credibility  gap  among  many  users  that  may  be  closed  by  a carefully  calculated 
series  of  flight  experiments  designed  to  verify,  one  step  at  a time,  the 
developments  of  recent  years.  A common  basis  for  evaluation  of  competing 
ideas  is  necessary.  Flight  experimentation  provides  exposure  to  the  flight 

| 

community  and  is  the  first  step  towards  acceptance.  This  step  is  necessary  — 
without  it  adaptive  control  research  will  likely  fade  away  as  it  had  in  the 
past.  Adaptive  control  is  too  promising  to  risk  the  fate  of  being  relegated 
to  the  status  of  a theoretical  tinker-toy. 

I would  caution  against  the  desire  for  an  exotic  application.  Although 
some  considered  the  F-8  airplane  application  to  be  less  successful  than  it 
might  have  been  because  the  F-8  had  acceptable  flying  qualities  and,  therefore, 
did  not  require  an  adaptive  system,  in  fact  the  F-8  seemed  a good  "teat  bed" 
and  the  argument  is  not  valid.  The  flying  qualities  of  any  airplane  can  be 


; 


* 
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improved  with  feedback.  This  objective  should  have  been  an  acceptable  one 
within  the  requirement  for  safety  of  flight.  The  purpose  was  not  to  replace 
the  original  F-8  flight  control  system,  but  to  show  that  an  adaptive  system 
can  work. 

Any  airplane  that  has  an  independent  controller  for  each  degree  of 
freedom  of  motion  and  measures  a full  state  vector  should  be  an  acceptable 
"test  bed."  If  it  is  required  that  the  subject  aircraft  have  poor  dynamics, 
this  can  be  done  to  any  aircraft  either  with  "hard-wired"  feedback,  by  adding 
lead  weight  to  the  tail  section,  or  by  other  means. 

There  seems  little  doubt  that  a select  series  of  flight  test  experiments 
at  this  time  will  do  more  to  not  only  define  the  future  directions  for 
adaptive  control,  but  also  to  create  a climate  of  acceptance  by  the  flight 


community. 


ADAPTIVE  CONTROL  OF  NON-MINIMUM  PHASE  PLANTS:  A REAL  PROBLEM1 


C.  Richard  Johnson,  Jr. 

Department  o£  Electrical  Engineering 
Virginia  Polytechnic  Institute  and  State  University 
Blacksburg,  VA  240S1 


As  currently  developed,  self- tuning  [l]  and  model  reference  adaptive 
controllers  [2]  rely  on  eventual  plant  numerator  cancellation  in  order  to 


achieve  their  established  control  objectives.  This  characteristic  has  limited 


stability  studies  and  applications  of  adaptive  control  to  plants  with  stable 


inverses,  typically  denoted  as  non-minimum  phase  plants.  (As  an  aside,  note 


that  the  non-minimum  phase  designation  arising  from  frequency  response 
magnitude/ phase  relationships  [3]  actually  requires  all  plant  singularities 
both  poles  and  zeros,  to  be  stable.  Common  usage  has  narrowed  this  tetm  to 


description  of  zero  locations.)  The  number  of  physical  plants  with  non' 


minimum  phase  behavior,  in  aerospace  and  industrial  process  applications  among 


others,  makes  this  narrow  focus  a real  problem 


Currently  four  approaches  exist  which  enjoy  limited  success  in  adaptive 


control  of  non-minimum  phase  plants:  (!)  simultaneous  identification  and 


control  (SIC)  (also  termed  indirect  adaptive  control),  (ii)  input  matching 


(IM),  (iii)  adjustable  model  reference  adaptive  control  (AMRAC),  and  (iv) 


delayed  model  control  (DMC) 


SIC:  Convergence  justification  of  all  SIC  schemes  relies  on  sufficient 


identifier  excitation  for  exact  plant  parameter  estimation.  This  restriction 


also  requires,  in  general,  plant  model  minimality  for  unique  parameterization 


Despite  these  two  restrictions  such  indirect  SIC  schemes  currently  enjoy  the 


broadest  possibility  of  control  objectives  restricted  only  by  what  controller 
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design  can  be  achieved  on-line  in  real  time  from  plant  parameter  estimates. 

For  example,  a SIC  scheme  requiring  on-line  factorization  has  been  proposed 
[4].  Recently  a significant  breakthrough  [5]  in  the  stability  analysis  of  the 
separation  basis  of  adaptive  observation  and  state  feedback  has  advanced  the 
possibility  of  SIC  provability.  Extension  of  these  separation  results  to  SIC 
has  been  proposed  [6]. 

IM:  Currently  IM  [7]  rests  on  single-stage  quadratic  cost  function  mini- 
mization (as  do  equivalent  techniques  arising  from  similar  but  distinct  con- 
cepts [8]  [9])  permitting  control  of  some  but  not  all  non-minimum  phase  plants 

[10] .  The  global  stability  of  adaptive  IM  is  currently  unproven  due  to  the 
reliance  of  three  recently  developed  discrete-time  stability  proofs  [ll][12] 
[13]  on  plant  inverse  stability.  The  extension  of  IM  to  a longer  horizon 
quadratic  cost  function  appears  to  require  further  plant  numerator  parameter 
knowledge  than  the  leading  parameter  knowledge  currently  deemed  necessary. 
Ongoing  studies,  e.g.  [14],  of  the  full  flexibility  of  single-stage  cost 
functions  may  improve  the  stabilizability  afforded  via  judicious  modifications. 

AMRAC:  The  concept  of  AMRAG  is  to  limit  the  adaptive  controller  cancellations 
of  plant  singularities  to  stable  values  and  adjust  the  reference  model  to  in- 
corporate the  unalterable  unstable  values.  All  adaptions  are  to  be  based  on 
the  error  between  the  model  and  plant-controller  outputs.  The  logical  AMRAC 
approach  has  apparently  borne  fruit  in  pole  cancellation  and  replacement  [15] 
and  pole-shifting  via  state  feedback  [16]  applications.  The  two  current  major 
drawbacks  of  extension  appear  to  be:  (i)  restriction  to  stable  plants  and 

(11)  possible  unstable  cancellations  in  most  general  controller  structures. 

DMC:  Recognition  of  the  effective  delay  inherent  in  non-minimum  phase 
responses  prompted  the  inclusion  of  a bulk  delay  in  the  reference  model  to  be 
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followed  in  order  not  to  try  and  undo  the  non-minimum  phase  effect  at 
tremendous  control  input  cost.  One  application  of  this  approach  has  been 
suggested  [17]  by  cascading  a MA  controller  and  a delayed  AR  model  of  a non- 
minimum phase  ARMA  plant.  The  delayed  AR  model  allows  compensation  of  the 
non-minimum  phase  zeros  of  the  plant  by  cancelling  after  time-shifting  the 
plant's  decaying  two-sides  AR  model  impulse  response.  Such  a scheme  does  not 
presently  appear  to  lend  itself  to  the  inclusion  of  feedback  control  elements. 

Further  evaluation  of  these  concepts  or  development  of  new  ones  to 
address  the  non-minimum  phase  plant  control  problem  represents  one  of  the 
most  immediate  tasks  of  adaptive  control  research. 
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ON  ADAPTIVE  CONTROL 
Bernard  Friedland 

The  Singer  Company,  Kearfott  Division 
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In  a round  table  session  on  adaptive  control  held  recently  at  a national 
conference  I observed  that  it  is  not  possible  on  the  basis  of  an  examination 
of  a control  system  to  determine  whether  or  not  it  is  adaptive.  This  is  a 
logical  impossibility,  nor  merely  a practical  problem.  In  order  to  determine 
whether  a system  is  adaptive  it  is  necessary  to  know  not  only  the  procedure 
used  by  the  designer,  but  also  the  terminology  he  used. 

The  very  use  of  feedback  in  the  design  of  a control  system  has  the  effect 
of  reducing  the  sensitivity  of  the  system  to  external  disturbances  and  to 
changes  in  characteristics  of  the  process.  The  behavior  of  the  system  tends  to 
be  invariant  to  changes  in  itself  or  in  the  environment  and  hence  it  is 
legitimate  to  call  such  a system  "adaptive"  in  the  customary  usage  of  that 
word. 

In  the  context  of  control  theory,  however,  the  usage  of  the  word  "adaptive" 
is  usually  defined  in  such  a way  as  to  exclude  control  system  designs  in  which 
the  "state  variables"  or  related  dynamic  variables  are  measured,  but  in  which 
the  "parameters"  are  assumed  known.  But  the  distinction  between  state  variables 
and  parameters  is  a convenience  of  the  control  system  designer  and  not 
necessarily  an  objective  fact.  For  example,  in  the  process 

x ■ - px  + u (1) 

everyone  would  call  the  variable  x the  "state"  and  the  variable  p the  "para- 
meter." But  suppose  p is  not  exactly  constant:  say. 


p • xv 
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where  v is  another  input  variable  (either  a control  or  a disturbance).  Now 
the  analyst  has  two  options: 

(A)  He  can  design  an  adaptive  control  system  for  (1)  by  either  tracking 
p or  by  making  the  design  robust  (so  that  a reasonable  range  of  variation  of  p 
can  be  acconmodated) . 

(B)  He  can  design  a non-adaptive  control  system  for  the  process  con- 
sisting of  (1)  and  (2)  together. 

An  individual  who  examines  the  resulting  system  should  have  no  trouble 
gauging  how  well  it  performs  but  would  have  no  way  of  knowing  whether  the 
system  is  adaptive. 

The  mechanism  of  feedback  provided  all  the  adaptation  needed  in  technology 
until  very  recently  (i.e.,  until  after  World  War  II)  when  availability  of 
sophisticated  electronics  (analog  and  digital  computers)  pointed  to  the 
possibility  of  improving  system  performance  at  the  expense  of  Increasing  hard- 
ware complexity. 

The  earliest  requirement  for  adaptive  control  arose  in  the  technology  of 
military  aircraft  flight  control  where  maneuverability  requirements  of  aircraft 
can  be  achieved  only  at  Che  expense  of  open- loop  dynamics  that  verge  on  in- 
stability. A feedback  control  system  can  improve  dynamic  performance  but  is 
sensitive  to  vehicle  parameters.  A very  reasonable  engineering  approach  is  to 
design  the  control  system  parameters  to  be  known  functions  of  the  vehicle  para- 
meters, and  to  track  the  latter  in  flight,  thereby  keeping  the  control  system 
"in  tune”  with  the  actual  aircraft  parameters.  Many  of  the  desired  parameters 

(i.e.,  "stability  derivatives")  are  reasonably  systematic  functions  of  a small 

2 

number  of  fundamental  variables,  primarily  dynamic  pressure  q - pv  / 2,  so  a 
good  deal  of  adaptivity  can  be  provided  by  simply  measuring  dynamic  pressure 
and  scheduling  the  control  system  gains  accordingly.  The  relative  simplicity 
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o£  this  technique  disqualifies  it,  in  the  eyes  of  some,  from  being  classified 
as  adaptive. 

At  the  other  extreme  is  the  "universal  bionic  controller"  (UBIC)  a box 
with  hundreds  of  input  and  output  terminals  and  with  a very  fast,  large-memory 
computer  inside  it.  No  a priori  knowledge  of  the  process  is  assumed,  signals 
from  everything  that  can  be  measured  are  connected  to  the  input  terminals,  and 
signals  from  the  output  terminals  are  brought  to  everything  that  can  be  moved. 

At  t ■ 0,  a start  button  is  pushed.  In  a short  time  the  controller  has  learned 
the  process  dynamics,  the  parameter  values,  and  has  determined  the  control 
parameters  of  the  optimum  control  law.  It  keeps  cycling  through  this  sequence 
so  that  parameter  changes  or  even  structural  changes  in  the  process  are  dis- 
covered and  the  control  law  is  recomputed  almost  instantaneously.  Perhaps 
such  a box  can  be  built,  but  those  of  us  who  are  around  to  see  it  work  will 
be  engaged  in  other  intellectual  pursuits. 

The  problems  that  merit  current  attention  are  somewhere  between  those  that 
can  be  solved  by  gain  scheduling  and  those  that  require  a UBIC.  Dynamics  of 
processes  in  this  category,  I would  think,  are  those  for  which  there  is  a 
general  concensus  about  which  variables  are  state  variables  and  which  are  para- 
meters. And  the  form  of  mathematical  model  that  represents  the  process  is 
reasonably  well  understood.  Moreover,  it  should  be  generally  agreed  that  the 
application  justifies  adaptive  control  because  the  process  is  either  difficult 
to  control  by  less  sophisticated  methods,  as  is  the  case  with  high-performance 
aircraft,  or  because  the  economic  benefits  of  slight  improvements  in  performance 
justify  the  outlay  for  an  adaptive  control  system. 
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ON  STOCHASTIC  ADAPTIVE  CONTROL 
C.  S.  Padilla 

Venezuelan  Institute  for  Scientific  Research  (IVIC) 

Caracas,  Venezuela 

When  we  want  to  design  control  laws  for  stochastic  systems  with  unknown 
parameters,  disturbance  inputs  and  measurement  noise,  two  roles  of  the  control 
input  have  to  be  considered.  The  first  one  is  to  excite  the  system  to  produce 
outputs  which  are  associated  with  desirable  system  performance,  and  another 
is  to  produce  outputs  from  which  good  estimates  of  the  unknown  parameters  can 
be  obtained.  The  kind  of  controllers  that  takes  into  account  these  two  roles 
is  called  a dual  controller.  To  find  the  optimal  controller  a closed  loop 
controller  has  to  be  designed  because  it  anticipates  that  future  measurements 
will  be  taken  and  it  achieves  a compromise  between  the  control  and  the 
estimation  objectives.  Due  to  the  fact  that  the  solution  to  the  optimal 
control  problem  is  computationally  impossible,  various  suboptimal  algorithms 
have  been  developed  that  have  the  dual  property.  Some  use  explicitly  in  the 
cost  function  the  conflicting  characteristic  of  the  control  [1-4]  and  others 
are  approximations  of  the  dynamic  programming  equation  and  lead  to  a cost 
function  where  the  conflicting  characteristic  of  the  control  is  evident  [5]. 

In  the  first  classification  we  include  the  safer  control  which  not  only  takes 
into  account  the  conflicting  characteristic  of  the  control  in  the  cost 
function  but  also,  through  the  design  of  an  output  sensitivity  weighting 
matrix,  redistributes  the  estimation  effort  according  to  the  accuracy  required 
to  achieve  a given  control  objective.  In  this  method  we  find  a control  and  a 
sensitivity  weighting  matrix  that  will  tend  to  reduce  the  sensitivity  of  the 
output  with  respect  to  the  unknown  parameters,  while  tending  to  achieve  output 
regulation. 
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The  problem  with  the  dual  control  algorithms  developed  up  to  now  Is 
computational  complexity.  A desirable  direction  for  future  research  is  to 
seek  computational  simpler  dual  adaptive  control  algorithms  and  to  develop 
control  laws  that  enhance  estimation  for  a class  of  nonlinear  stochastic 
control  problems. 
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A MINIMAX  APPROACH  TO  THE  DUAL  CONTROL  PROBLEM 
Anthony  V.  Sebald 
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University  of  California,  San  Diego 
La  Jolla,  CA  92093 

Consider  the  following  statement  of  a dual  control  problem.  It  is 
desired  to  design  a closed  loop  control  for  the  system 

*k+l  " Ak(9)xk  + Bk^9>vk;  *0  * x(0);  k " (1) 

• V9>*k + 0k<9>“k  «> 

in  such  a way  as  to  minimize  the  maximum  of  the  incremental  quadratic  per- 
formance index 

R(u,0)  - J(u, 9)  - J(u*,0)  over  all  9 € © (3) 

where 

9 is  a vector  of  constant  but  unknown  parameters.  It  is  further 

p 

assumed  that  9 is  an  element  of  a known  compact  subset  9 of  R . 

9 may  be  convex  and  no  prior  probabilistic  information  on  9 
is  assumed, 
x and  y are  scalars. 

x(0)  is  a scalar  Gaussian  random  variable  whose  mean  and  variance  are 
known  functions  of  9. 

w is  a scalar-white  Gaussian  noise  process  whose  mean  and  covariance 

lx 

are  also  known  functions  of  9. 

v £ Cvi,v2» * * • ,vfl] ' 18  the  vector  of  controls  over  the  horizon  0 to 
N resulting  from  the  feedback  control  law  u(Y). 

Y is  the  measurement  set  [y^,y2> • • • >y^] ' • 

it 

Ug  is  the  control  law  which  minimizes  J(u,9)  for  known  9. 
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J(u,0)  £ j E{QlXJ  + E [Q^xJ  + Qjl)v^]  |e) 

A scalar  structure  is  posed  to  simplify  the  mathematical  exposition  Exten- 
sion to  the  vector  case  presents  no  essential  difficulties. 

It  is  worthwhile  to  carefully  consider  the  implications  of  this  problem 
statement.  In  the  first  place,  a feedback  control  of  the  form 

v - u(Y)  (4) 

is  desired.  Secondly,  no  plant  noise  is  allowed.  This  is  a useful  formula- 
tion in  many  control  problems  (e.g.  control  of  spacecraft  suffering  from 
launch  uncertainties).  While  presently  essential,  it  is  expected  that  the 
latter  restriction  will  yield  with  further  study.  Thirdly,  we  assume  that 
the  parameter  vector  9 is  known  only  to  lie  in  a closed  and  bounded,  possibly 

p 

convex  subset  of  fl  and  that  no  other  prior  information  is  available.  In 
particular,  Bayes  solutions  are  impossible  and  no  intelligent  adversary  is 
assumed  to  control  9.  Finally,  the  choice  of  performance  index  is  crucial. 

As  shall  be  demonstrated  below,  it  is  convenient  to  attempt  to  minimize  the 
maximum  performance  difference  between  a given  control  u and  the  optimal 
control  u*  which  could  be  used  if  9 were  known.  R(u,9)  is  computationally 
attractive  and  it  permits  formulation  of  the  problem  as  a minimax  problem 
without  requiring  choice  of  9 by  an  intelligent  adversary.  Consider  the 
qualitative  description  of  Fig.  1 for  a scalar  0 £ The  optimal 

controller  u*  satisfying  (3)  would  attempt  to  provide  a performance  J(u*,9) 
which  would  match  J(u*, 9)  as  closely  as  possible  over  the  entire  0.  It  would 
do  so  by  paying  a slightly  higher  price  than  u at  9 ■ 02  1®  return  for  a 
reduction  in  the  cost  for  u near  9 ■ 0^.  It  very  probably  would  not  equal 
J(u*,9)  for  any  9 6 0*  Using  a minimax  criterion  on  the  incremental  quadratic 


Figure  1.  A qualitative  comparison  of  the  performance  of  various 
controllers: 

u*  minimizes  J(u,9)  for  a fixed  6 
u is  minimax  for  J(u,9)  over  9 6 9* 
u*  is  minimax  for  R(u,9)  over  9 € 9* 


loss  function  R(u,9)  effectively  selects  that  controller  which  performs  as 


close  as  possible  over  all  9 to  the  best  one  could  do  with  complete  knowledge 


of  9 - a very  desirable  performance  characteristic.  Fortuitiously,  it  is  also 


computationally  tractable.  A similar  incremental  loss  function  has  been  used 


for  some  time  in  the  design  of  minimum  sensitivity  controls  for  deterministic 
systems  [1]. 


II.  The  Optimal  Solution 


For  several  reasons,  it  is  convenient  to  view  (l)-(3)  as  a game  between 


the  statistician  (who  chooses  u(Y))  and  nature  (who  chooses  9).  As  discussed 


above,  (3)  permits  the  use  of  a minimax  criterion  without  its  usual  pessimism 


162 

Secondly,  a great  deal  Is  known  about  the  use  of  game  theory  in  the  deter- 
mination of  minimax  solutions  to  problems  like  (l)-(3). 

A game  is  completely  specified  by  a pair  of  strategy  spaces  from  which 
each  player  extracts  his  action  and  a loss  function  which  determines  the  pay- 
off for  each  possible  pair  of  actions.  Assuming  the  loss  function  (3),  a 
minimax  statistician's  solution  to  the  game  would  satisfy: 

min  max  R(u,9)  (5) 

u 9 

Unfortunately,  this  optimization  is  difficult  since  the  minimization 

must  be  performed  over  a function  space  (the  statistician's  strategy  space). 

If  min  max  R(u,0)  - max  min  R(u,9),  and  if  min  R(u,9)  could  be  easily  deter- 
u 9 0 u u 

mined  for  a fixed  9,  then  the  optimization  (5)  becomes  a much  simpler  para- 
metric one.  The  first  condition  holds  if  the  game  has  a value.  The  second 
holds  if  the  minimax  solution  to  the  game  is  equal  to  the  Bayes  solution  under 
some  computable  prior.  The  latter,  if  it  exists,  is  called  the  least  favor- 
able prior  for  the  game.  Finally,  of  course,  the  minimax  solutions  must  exist. 
Since  a well  behaved  game  has  all  of  these  properties,  the  solution  to  (l)-(3) 
will  be  achieved  if  it  can  be  cast  in  the  proper  framework. 

The  first  difficulty  is  to  determine  the  strategy  spaces  for  both  players. 

9 is  a reasonable  choice  for  nature.  A first  choice  for  the  statistician's 

T 

space  might  be  l(  A {u:  tr  E{v  v|q)  < • V 0 g 9}.  Since  neither  9 nor  H are 
finite  spaces,  the  game  (9,fy,R)  is  difficult  to  solve.  Also,  the  use  of  pure 
strategies  such  as  u and  9 do  not  always  guarantee  the  desired  properties. 

One  is  therefore  at  least  initially  forced  to  consider  randomized  strategy 
spaces. 

In  the  present  context,  a randomized  strategy  for  the  statistician  would 
be  the  choice  of  a probability  distribution  on  the  elements  of  the  given 
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function  space  rather  than  a particular  function  u(Y).  Similarly,  a random- 
ized strategy  for  nature  would  be  the  choice  of  a distribution  of  0 c 9 rather 
than  a choice  of  a specific  value  of  9.  Fortunately,  randomizations  on  the 
statistician's  space  will  not  be  necessary. 

In  the  sequel,  we  shall  define  a subset,  y^,  of  y which  is  weakly  com- 
pact and  demonstrate  that  R(u,@)  is  weakly  lower  semi-continuous  for  all 
u € This  choice  of  both  y^  and  the  weak  topology  are  the  key  to  insuring 

that  the  resulting  game  (©.“l^.R)  can  be  used  to  solve  (l)-(3).  After  intro- 
ducing some  notation,  this  result  will  be  formalized  in  a Theorem. 

Let: 

y A space  of  all  possible  observations  sets  Y * {y^:  i « 1,2,...,N}. 

y A space  of  all  functions  u:  y -1RN  such  that 

|lu(Y)||^  £ tr  Eb£Tv|9l  < ® v e e . 

^ A {u:  V “*1RN  IlkOOlL^  M < ® v 0 €0}. 

J(u,9)  A J(u,  0)  |x(0)a0  a.e. 

A [u:  1/  - RN  (u  € U and  J(u,  0)  < « V 9 € 0}. 

9 A space  of  allowable  values  of  0. 

0*  A space  of  all  probability  distributions  on  0,  i.e., 

{t(9):  0 € 8,  t is  a valid  probability  distribution  and  t(0)  ■ l}. 

space  of  all  probability  distributions  on  y^,  i.e., 

{y(u):  u € y^,  y is  a valid  probability  distribution  and 

Y(^>  - 1). 

R)  for  the  purpose  of  finding 
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r 


sup  E{R(u*,9)}<  sup  E fR(u,9)}  V u € U.  (6) 

T € 0*  T T € 0*  T ^ 

where  E^*}  denotes  expectation  with  respect  to  the  density  r*  The  minima* 
u*  and  several  of  its  crucial  properties  are  given  in  the  following  Theorem 
which  is  proved  in  [2]. 

Theorem  1: 

The  non- randomized  control  laws  form  an  essentially  complete 
class  and  therefore  randomized  control  laws  need  not  be  con- 
sidered. 

The  game  (O.l^.R)  has  a value. 

There  exists  a least  favorable  prior  Tn  € 0j  where  0*  is  the 

u a a 

space  of  all  discrete  probability  distributions  on  0 having  a 
finite  number  of  points  of  positive  support, 
u*  is  Bayes  with  respect  to  Tq. 

This  is  indeed  a very  remarkable  result.  It  demonstrates  that  the 
optimal  solution  to  the  reformulated  dual  control  problem  is  Bayes  with 
respect  to  a finite  dimensional  prior  even  though  it  is  both  multivariate 
and  convex. 

Theorem  1 guarantees  the  existence  of  both  the  minimax  controller  and 
the  least  favorable  prior  and  specifies  the  structure  of  the  optimal  con- 
troller thereby  converting  the  original  function  space  optimization  problem 
to  one  of  parameter  optimization.  In  particular,  since  (0,^,R)  has  a value, 
it  is  sufficient  to  determine  the  u € which  satisfies 

sup  inf  EfR(u,9)} 

T € 0d  u 6 Ifc 

which  by  Theorem  1 is  equivalent  to  finding  Tq  € 0^  for  which 


(i) 


(ii) 

(iii) 


(iv) 


Mote  that  provided  the  Bayes  control  is  available,  the  only  unknowns  in 


(6)  are  the  K points  of  positive  support  of  rn  and  the  K-l  probabilities 


which  rn  applies  to  them.  The  methodology  of  (6)  is  robust  since  it  is 


optimal  for  compact  parameter  spaces  and  for  all  LQG  systems  admitting  an 


optimal  controller  with  finite  loss  for  all  9 € 0 
Corollary  1;  u*  is  approximately  given  by  [3]: 


where 


P(0.)m. , the  optimal  solution  to  the  LQG  problem  for  9 ■ 0 


9.  are  the  points  of  positive  support  of  rn  6 ©' 


....x^]'  the  minimum  mean  square  error  (MMSE) 

estimate  of  x given  9 ■ 9. ; 


P(0  ) is  the  optimal  controller  gain  for  the  LQG  problem  with  9 ■ 9 


j 9 )p  *s  t*ie  8ener®Hzed  likelihood  ratio; 

dF(Y  |9)  is  the  conditional  probability  measure  of  Y given  9 
p.  is  the  probability  mass  given  9 ■ 0.  by  t». 


1)  In  the  design  of  adaptive  controllers,  an  Incremental  quadratic  loss 


function  is  useful  in  that  it  permits  use  of  no n- pessimistic  mlnimax 


techniques 
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2)  It  has  been  demonstrated  using  Hilbert  space  and  game  theoretic  techni- 
ques that  the  optimal  solution  to  the  dual  control  problem  (l)-(3)  is 
Bayes  with  respect  to  a least  favorable  prior  having  a finite  number  of 
points  of  positive  support.  Furthermore,  it  has  the  same  structure  as 
the  standard  Bayes  controller  for  J(u,0)  of  (3). 

3)  The  resulting  optimal  structure  lends  itself  readily  to  suboptimal 
solutions. 

4)  Results  contained  herein  significantly  reduce  the  class  of  Bayes  problems 
of  relevance  to  optimal  dual  control  problems. 
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Comment  1 

In  addition  to  the  need  for  continued  support  of  research  in  the  more 
established  areas  of  stochastic  adaptive  control,  I feel  that  it  is  also 
important  that  the  Air  Force  support  research  in  the  area  of  decentralized  and 
hierarchical  adaptive  control — for,  from  this  research  could  arise  important 
new  classes  of  self  organizing  control  systems. 

Comment  2 

The  control  community  needs  a generic  class  of  problems  at  different 
levels  of  complexity  that  potentially,  at  least,  require  algorithms  based  upon 
stochastic  adaptive  control  theory.  This  class  of  problems  should  exhibit  a 
diversity  of  complexity  with  respect  to  specified  performance  measures. 

A set  of  evaluation  procedures  should  be  precisely  specified.  In 
addition  to  quantifying  conventional  measures  of  performance,  these  procedures 
should  quantify  how  matters  such  as  robustness  to  various  parameter  sets, 
operating  regimes,  nonlinearities,  etc.,  are  to  be  explicitly  evaluated.  The 
evaluation  procedure  should  also  require  a quantitative  assessment  of  the 
resulting  control  algorithms  relative  to  their  (1)  computer  hardware  and  soft* 
ware  requirements  (in  terms  of  some  given  generlcally  defined  computer  system), 
and  (2)  their  simplicity  per  se  as  well  as  their  ease  of  use  by  control 
practitioners  other  than  the  author (s)  of  the  algorithms. 

Wherever  possible,  simple  benchmark  algorithms  should  be  suppliad. 


The  problems  should  be  based  upon  important  practical  problems,  but 
should  be  stated  in  such  a manner  that  virtually  any  member  in  the  community 


of  control  researchers  could  address  the  problem  of  possibly  contributing  an 


algorithm  to  the  "competition."  Each  possible  contributor  would  then  start 


with  essentially  the  same  state  of  knowledge  relative  to  the  problem  state 


ment  and  the  quantatively  defined  evaluation  criteria 


These  requirements,  while  admittedly  being  classif icable  as  "difficult 


are  well  within  the  capabilities  of  the  control  community.  The  expected 
benefits  of  such  a program  make  it  well  worth  the  effort  required  to  establish 
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MACROECONOMIC  POLICY  MODELING  AND  ADAPTIVE  CONTROL 
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Department  of  Economics 
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Los  Angeles,  California  90007 

Abstract 

Lucas  [6]  has  recently  argued  that  government  policy  planning 
undertaken  with  traditional  macroeconomic  policy  models  is  un- 
reliable, since  no  allowance  is  made  in  these  models  for  the  re- 
action of  other  rational  decision-making  agents  in  the  economy  to 
changes  in  government  policy.  However,  the  incorporation  of 
synanetrical  rationality  into  macroeconomic  policy  models  leads  to 
a difficult  adaptive  control  problem  for  government  planners 
requiring  the  on-line  estimation  of  control  and  state  dependent 
coefficients.  Three  alternative  approaches  to  the  problem  are 
briefly  mentioned. 

Consider  the  following  macroeconomic  policy  model  format,^  typically  used 
([!]>  [3],  [4])  to  describe  the  policy  choice  problems  facing  government 
planners.  The  observed  motion  of  an  economic  system  over  N periods  is  described 
by  a difference  equation 

x^  » x (initial  conditions)  , (la) 

Vi  ■ Wv  *„>•  1 £ai  * • <lb> 


^Macroeconomics  is  the  study  of  major  economic  aggregate  variables  such 
as  total  production  (GNP),  total  employment,  the  average  price  level  of  all 
goods  and  services,  and  the  total  money  supply.  Macroeconomic  investigations 
generally  focus  on  two  important  concerns:  The  causal  relationships  among  the 
aggregate  variables;  and  the  prediction  of  the  effects  on  these  aggregate 
variables  of  alternative  government  agency  policies  (control  actions).  Models 
designed  for  the  latter  purpose  are  referred  to  as  macroeconomic  policy  models. 
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1 


g 

where  the  nth  period  system  state  is  an  element  of  R , the  nth  period 
government  control  vn>  to  be  announced  at  the  beginning  of  period  n,  is  con- 

C T 

strained  to  lie  in  an  admissible  control  set  V(n,x  ) contained  in  R , u € R 

1 n n 

is  a random  vector  composed  of  unknown  parameters  in  equation  (lb)  together 

3T^l€"t*8  3 

with  a residual  error  term,  and  the  state  function  fQ:  R — R is  con- 

tinuous. The  value  associated  with  each  possible  configuration  (uu  , v , x ) 

q n n 

for  period  n is  measured  by  a continuous  return  function  Wr:  R — R. 

Finally,  the  set  £ of  admissible  feedback  control  laws  for  system  (1)  consists 

8 C 

of  all  vectors  v » (v^(«)» • • •»vjj(*))  of  measurable  functions  vq:  R - R 
satisfying  vq(x)  € V(n,x)  for  each  n and  x. 

The  traditional  approach  to  macro  policy  modeling  assumes  that  each 
random  vector  u)q  is  an  independent  drawing  from  a possibly  degenerate  prob- 
ability distribution  (Rr,  B,  pq)  that  is  known  (estimated)  by  the  planner 
prior  to  period  1.  The  planner  in  period  1 thus  faces  an  ordinary  stochastic 
control  problem:  Maximize  expected  total  return 

N _ 

E[  £ W (u>  , v (x  ),  x ) lx]  (2) 

L . nv  n’  n'  a' n'  1 J 
n-1 

subject  to  (1)  by  selection  of  feedback  control  law  v € £• 

Recently,  however,  it  has  been  emphasized  by  Lucas  [6]  and  others  that 
this  traditional  approach  to  macroeconomic  policy  modeling  ignores  important 
game  aspects  inherent  in  economic  planning.  Realistically,  the  random  vectors 
<vn  in  (lb)  should  be  Interpreted  as  functions  of  the  demand  and  supply 
decisions  of  other  optimizing  agents  in  the  economy  who  react  rationally  to 
changes  in  government  policy  and  the  state  of  the  economy.  Specifically, 
letting  Ir  denote  the  nth  period  Information  set  consisting  of  past  and  current 
control  and  state  realizations  vn  a (v^,...,vn)  and  xa  s (x^,...,xn),  in 
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addition  to  the  relevant  system  dynamics  (1),  the  nth  period  random  vector 
(Un  might  reasonably  be  assumed  to  be  governed  by  a more  general  Stackelberg 
probability  distribution  of  the  form  (Rr,  B,  p(*  |*n))* 

If  the  distributions  p(*  |ln)  are  known  to  the  government  planner  in 
period  1,  then  in  principle  he  still  faces  an  ordinary  stochastic  control 
problem.  However,  although  it  is  conceivable  that  satisfactory  control  per- 
formance might  be  achieved  over  periods  [l,...,N]  following  the  prior  off-line 
estimation  of  distributions  Pn(*)  exhibiting  only  time  dependencies  (e.g., 
stable  periodicities),  it  is  entirely  inconceivable  that  general  distributions 
of  the  form  p(*  jl^)  could  be  estimated  off-line,  prior  to  any  control  action 
by  government.  Rather,  the  government  planner's  assignment  must  now  realisti- 
cally be  viewed  as  a difficult  on-line  adaptive  control  problem  involving 
simultaneous  learning  and  control,  a task  for  which  traditional  macro- 
econometric estimation  techniques  are  ill-suited. 

In  Refs.  [8-9]  an  adaptive  control  method  is  developed  which  is  applicable 
to  this  problem,  assuming  the  existence  of  (unknown)  Markov  transition  prob- 
abilities p(»  |lQ)  = pfl(*  )vn>xn)  and  the  observability  of  the  realizations  u)n. 

The  key  distinguishing  feature  of  the  method  is  the  direct  estimation  and  up- 
dating of  the  relevant  dynamic  programming  optimality  equations  in  each  period 
n without  resort  to  explicit  probability  distribution  specification. 

An  alternative  game- theoretic  approach  is  suggested  in  [10]  and  developed 
in  [5]  in  the  context  of  a C (command,  control,  and  communication)  model  having 
the  basic  format  (1)  and  (2).  The  vectors  u>n  in  (lb)  are  interpreted  as  the 
realization  for  an  unknown  feedback  control  law  u>  implemented  by  an  opposing 
player.  A Markov  transition  probability  pn(*  |xr)  Is  generated  for  uu^  on  the 
basis  of  a probabilistic  assessment  over  opposing  player  preference  structures 
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and  an  iterative  derivation  for  the  (unique)  Nash  equilibrium  strategy  pair 
(v,uu)  corresponding  to  each  preference  specification. 

Nevertheless,  both  of  these  approaches  are  open  to  criticism  in  macro- 
economic  contexts,  the  first  due  to  reliance  on  observable  realizations  uu  . 

' Q 

and  the  second  due  to  implicit  reliance  on  the  existence  of  a single  (or 
representative)  opposing  player. 

Macroeconomists  have  recently  begun  to  explore  a third  alternative 

involving  prior  linear  restrictions  on  the  form  of  x , v , and  uu  (see  [2]  and 

n n n 

[7]).  For  Illustration,  suppose  the  state  equations  (lb)  take  the  linear  form 
xq+1  - Axq  + u)n  + Cvn,  n € £1,  — ,N}  , (3) 

where  A and  C are  known  constant  coefficient  matrices,  and  suppose  government 

is  restricted  to  linear  control  laws  of  the  form  v (x  ) * G x . Finally, 

n ci  q n 

letting  E[*  |ln]  denote  expectation  with  respect  to  p(*  |ln)>  suppose  uu^  takes 
the  form 

\ * W + * *»  • <4> 

where  B.  and  B_  are  known  coefficient  matrices  and  e Is  a white  noise  zero- 
l l a 

mean  process.  The  interpretation  of  (3)  and  (4)  is  that  the  behavior  uu  of 

n 

other  agents  in  the  economy  enters  linearly  into  the  state  equation  (3)  in  the 
form  of  state  expectations  which  are  "rational"  in  the  sense  that  they  are 
consistent  with  the  true  state  generating  mechamism  (3).  Substituting  for  vq 

and  oun  in  (3),  and  taking  expectations  with  respect  to  E[*  |lnL  a € £l N}, 

the  expectations  in  (4)  can  be  eliminated  by  backward  recursion.  The  resulting 
reduced  form  state  equation  takes  the  form 

xn+l  ■ <*«„)»„  * Cl-»1]'lBjq(Gn).|>.1  + , (5) 


where  I denotes  the  identity  matrix  and 

Q(Gn)  = [I  - Bx  - B2]-l[A  + CGn]  . (6) 

The  rational  expectations  approach  assumes  that  agents  in  the  economy 
know  the  "true  structure  of  the  model,"  i.e.,  the  structures  (3)  and  (4), 
including  exact  knowledge  of  the  coefficient  matrices  A,  , B2.  and  C.  On 
the  other  hand,  it  is  recognized  [6]  that  government  planners  will  typically 
have  to  estimate  A and  C in  (3)  and  B^  and  B^  in  (4).  The  exact  functional 
form  of  Q(«)  in  (6)  would  then  have  to  be  estimated,  presumably  on-line. 
Traditional  econometric  techniques  are  not  directly  applicable  to  this  task. 

In  Ref.  [11]  it  is  suggested  that  Kalman  filtering  techniques  might  be  used. 

Many  critics  have  faulted  the  rational  expectations  assumption  that  agents 
other  than  government  have  perfect  knowledge  of  the  true  structural  model. 
However,  a second  major  difficulty  with  the  rational  expectations  approach 
which  has  not  received  as  much  attention  is  that  specifications  such  as  (4) 
for  the  behavior  of  "rational"  economic  agents  are  ad  hoc,  since  they  are  not 
derived  from  the  underlying  optimization  problems  (e.g.,  profit  maximization) 
which  actually  face  these  agents. 
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Uncertainty  of  aircraft  models 
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J.  Ackermann 

Parameter  space  design  of  robust  control 

7) 

D.  Looze 

Optimization  approach  for  robust  control 

8) 

R.  Marsh 

Problems  in  unstable  aircraft 

II.  Discussions 

Impulse  response  models  of  the  plant  --  although  not  a minimal  repre- 
sentation --  offer  computational  advantages  because  the  output  (convolution 
sum)  is  bilinear  in  input  and  impulse  response,  i.e.  for  a given  input  linear 
in  the  model  coefficients. 

Update  reference  trajectory  during  operation  to  avoid  stationary  error. 
Computations  with  truncated  impulse  response  (e.g.  after  20  significant 
steps),  i.e.  only  for  stable  plants.  Also  other  techniques  assume  open  loop 
stability. 

Combinations  of  adaptive  and  fixed  gain  robust  control: 

1.  Use  a fixed  gain  robust  controller  mainly  for  stabilization,  then 
improve  performance  by  adaptive  control. 

2.  Use  a fixed  gain  robust  controller  as  backup  for  the  case  of  a failure 
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in  Che  adaptive  system  or  in  a gain  scheduling  system.  Air  data  measurements, 
e.g.  dynamic  pressure,  are  not  very  reliable. 

3.  Under  external  noise  an  adaptive  system  may  not  adjust  fast  enough 
to  a fast  change  in  plant  parameters  (e.g.  drop  of  a load,  variation  of  the 
geometry,  hurt  in  fight).  Switch  to  a fixed  gain  robust  system,  until  the 
identification  has  followed  and  adaptation  can  improve  the  performance. 

4.  Adaptive  control  theory  usually  does  not  deal  with  problems  of 
structural  identification  (e.g.  failure  detection)  and  structural  adaptation 
after  a failure  has  been  detected.  However  problems  are  related:  Fast 
structural  identification  may  lead  to  false  alarms,  in  particular  under  noisy 
conditions.  Slow  and  reliable  structural  identification  may  leave  the  system 
in  a failed  unstable  configuration  for  a while.  The  control  should  be  de- 
signed to  provide  robustness  of  stability  with  respect  to  the  failure,  then 
nothing  very  bad  happens  until  the  failure  is  detected  reliably. 


5.  Robust  fixed  gain  control  may  be  combined  with  some  redundancy  con- 
cepts. Various  levels  are  possible: 

a)  Passive  redundancy  by  paralleled  components.  The  50Z  gain  reduction 
margin  of  LQ  designs  offers  the  possibility  to  use  two  paralleled 
sensors  or  actuators,  such  that  in  case  of  a failure  the  gain  is 
reduced  only  by  50Z. 

b)  Removal  of  failed  components.  Even  if  a component  failure  can  be 
tolerated,  as  far  as  stability  is  concerned,  it  may  be  necessary  in 


the  long  run  to  remove  a failed  component,  e.g.  to  close  a leaking 
gas  jet  valve  by  a safety  valve  or  to  remove  a bias  term  entering 
into  a control  system  from  a sensor  failed  at  a nonzero  constant 
value. 
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c)  Analytic  redundancy  may  help,  if  an  adaptive  observer  provides  an 
estimate  for  a missing  signal. 

d)  Hardware  redundancy,  e.g.  majority  voting  in  a multiplexed  system 
can  bring  the  system  back  to  its  original  performance.  However  this 
part  of  the  system  ideally  should  not  be  vital  for  stability,  see  4.). 

6.  Examples  show  that  a wide  range  of  parameter  variations  can  be 
accomodated  by  a fixed  gain  robust  controller,  provided  only  physically 
reasonable  requirements  are  made,  in  particular:  Do  not  try  to  make  a slow 
system  fast  or  a fast  system  slow.  Not  one  reference  model,  but  fast  and  slow 
reference  models,  for  different  operating  conditions.  Or  in  robust  control: 
Invariance  only  of  damping  or  maximum  overshoot,  not  of  natural  frequency  or 
time  of  maximum  overshoot.  Of  course  it  depends  on  the  application  which 
property  must  be  robust:  In  the  design  of  an  oscillator  the  frequency  must 
be  constant,  in  the  design  of  a crane  the  frequency  of  oscillation  is  un- 
important. 

If  robust  control  cannot  cover  the  whole  range  of  parameter  variation, 
it  may  be  necessary  to  use  two  or  more  fixed  gain  controllers  and  gain 
scheduling  based  for  example  on  a dynamic  pressure  measurement  or  other  crude 
classification  of  flight  conditions.  Here  the  question  of  robustness  of  gain 
scheduling  in  view  of  the  use  of  not  very  reliable  air  data  arises.  Thus  it 
may  be  a design  goal  to  have  a wide  overlapping  range  of  flight  conditions, 
for  which  either  set  of  gains  gives  satisfactory  stability,  such  that  the 
switching  condition  is  not  critical.  We  have  discussed  just  the  first  step 
towards  adaptation,  with  each  additional  step  additional  robustness  problems 
will  arise. 
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The  Man  and  the  Machine 


1.  Some  requirements  on  the  control  system  are  typical  for  situations 


where  a man  is  controlling  the  outer  loop.  A man  can  control  also  an  unstable 


plant  provided  the  eigenvalues  in  the  right  half  plane  are  close  to  the 


origin.  He  has  more  problems  if  he  has  to  control  fast  modes,  even  if  they 


are  slightly  damped.  In  other  words,  the  stability  boundary  is  not 


necessarily  the  best  emergency  boundary  for  sensor  failures,  it  may  be  some 


thing  like  the  boundary  in  Fig.  1 


Fig.  1.  Emergency  boundary  for  sensor  failures  in  situations 
with  a man  in  the  outer  loop. 


2.  The  problems  of  actuator  and  sensor  failures  look  similar,  if  we 


interpret  them  as  a row  or  a column  of  the  feedback  matrix  being  switched  to 


zero.  However  man  can  use  his  sensors  (eyes,  feeling  for  accelerations)  as 


i.e.  for  him  the  aircraft  remains  observable  under  sensor  failures 


however  it  may  become  uncontrollable  under  actuator  failures 


3.  The  pilot  does  not  want  to  be  a passenger.  He  may  want  to  identify 


the  controlled  aircraft  by  "playing"  with  the  input  signals.  Control  schemes 


which  give  him  the  same  feeling  for  a wide  range  of  parameter  variations,  may 


be  dangerous,  if  the  dynamics  suddenly  become  bad  beyond  an  assumed  range  of 


parameter  variation.  The  pilot  needs  a warning  before  the  "cliff 


another  reason  why  the  dynamics  should  change  with  changing  parameters,  i.e 
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for  different  operating  points  different  reference  models  should  be  assumed. 
Previously  we  had  discussed  this  point  only  under  the  aspect  of  control  and 
control  rate  limitations. 

4.  The  human  is  a natural  example  for  the  solution  of  robustness 
problems  for  sensors  and  actuator  failures.  The  fact  that  we  have  two  eyes 
and  two  ears  gives  us  the  additional  capability  of  stereo  vision  and  hearing 
in  the  unfailed  case.  In  a discussion  we  use  both  voice  and  hands  (e.g. 
writing  on  a blackboard)  to  communicate,  although  we  could  still  communicate 
if  one  of  these  actuators  fails.  It  may  be  conjectured  that  the  idea  of 
having  standby  components,  which  do  not  contribute  to  the  nominal  performance 
in  the  unfailed  case, are  typical  only  for  man-made  systems  (e.g.  redundant 
components,  spare  tire). 

5.  Also  the  designer  is  a human.  Control  theory  should  provide  him 
with  convenient  tools,  e.g.  for  the  computer-aided  design  of  control  systems, 
instead  of  demanding  that  the  designer  has  to  put  all  thinkable  tradeoff 
situations  into  one  scalar  performance  index  or  set  of  inequalities. 

A general  conclusion  by  the  participants  was  that  this  type  of  workshop 
with  participation  of  practitioners  and  theoreticians  is  very  helpful  for  the 
mutual  understanding  and  cooperation  of  both  sides.  The  practitioners 

! 

emphasized  and  specified  their  need  for  robust  control. 


I 

I - 

I 
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Report  of  the 

WORKING  GROUP  ON  MODEL  REFERENCE  ADAPTIVE  CONTROL 
AND  STOCHASTIC  SELF-TUNING  REGULATORS 

Discussion  Leader:  I.  D.  Landau 


The  following  subjects  were  discussed: 

1.  Deterministic  assumptions  for  the  design  of  MRAC. 

2.  Stochastic  assumptions  for  the  design  of  STURE. 

3.  Stability  and  convergence  problems  for  deterministic  and  stochastic 
adaptive  control. 

4.  Transients  of  the  adaptation  processes. 

5.  Explicit  and  Implicit  MRAC. 

6.  Assumptions  upon  the  leading  coefficient  of  the  plant  transfer 
function. 

7.  Adaptive  control  of  non-minimum  phase  plants. 

8.  Use  of  reduced  order  models. 

9.  Suggestions  for  theoretical  research. 

10.  Suggestions  for  applications  of  MRAC  and  STURE. 

We  next  summarize  for  each  of  these  subjects  the  discussions  and  con- 
clusions of  the  working  group. 

I.  Deterministic  Assumptions  for  the  Design  of  MRAC 

Consider  that  the  plant  to  be  controlled  is  characterized  by  the  transfer 


function: 


gctp(s) 


TP(S)  " 0-(s) 


Op (s)  • 1 + OjS  + , ..  + OjjS 

np 

0p(s)  - 1 + 0ji  + ...  + 0as 


I 


I 


i 

: 


i 

i 
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The  following  assumptions  are  considered  for  the  design: 

1)  Knowledge  of  a bound  n: 

n > - deg  0p(s)  . 

2)  Exact  knowledge  of  n in  the  continuous  case: 

n*  - deg  0p  - deg  Op  - np  - mp  . 

'tc 

In  the  discrete  case,  the  plant  delay  is  the  equivalent  of  n and 
should  be  known. 

3)  The  design  is  based  on  input-output  data  only. 

4)  The  design  does  not  use  differentiators  in  the  continuous  case 
or  predictors  in  the  discrete  case. 

5)  The  plant  is  assumed  to  be  minimum  phase. 

6)  The  design  provides  only  infinite  time  convergence  results. 

7)  Some  a priori  information  upon  the  gain  "g"  is  necessary  (for 
further  details,  see  subject  6). 

II.  Stochastic  Assumptions  for  the  Design  of  STURE 

In  addition  to  the  assumptions  considered  in  the  deterministic  case,  the 
following  assumptions  are  specific  for  the  stochastic  case: 

1)  Existence  of  disturbances. 

2)  The  colour  of  disturbances. 

3)  Only  second  order  properties  are  necessary  for  design. 

Some  particularities  have  been  enhanced  namely: 

• In  the  stochastic  context,  the  poles  of  the  observers  (or  state 
variable  filters)  are  adapted  since  their  optimal  values  will  depend  on  the 
colour  of  the  disturbance.  This  is  not  the  case  for  MRAC. 

• Large  changes  in  parameters  lead  to  violent  transients. 
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III.  Stability  and  Convergence 

In  the  context  of  MRAC  and  STURE,  the  term  stability  roughly  means  that 
all  the  quantities  are  bounded.  Convergence  means  that  in  the  deterministic 
context,  the  plant-model  error  goes  to  0 a«  t - ■ and  in  the  stochastic  con- 
text, convergence  means  that  the  adjustable  parameters  converge  in  prob- 
ability to  the  desired  values. 

Recent  research  results  have  established  several  procedures  for  proving 
stability  and  convergence  of  MRAC  and  STURE  and  these  procedures  have  been 
already  applied  to  analyze  several  typical  schemes. 

These  proofs  have  emphasized  the  differences  between  discrete  time  scheme 
and  continuous  time  scheme.  These  differences  came  essentially  from  the 
following: 

• In  the  continuous  time  case,  the  output  of  a minimum  phase  asymptoti- 
cally stable  linear  block  can  be  bounded  with  the  input  being  unbounded. 

• In  the  discrete  time  case,  the  boundedness  of  the  output  of  a ml  n't  mum 

1 

phase  asymptotically  stable  linear  block  implies  the  boundedness  of  the  input. 

> j 

IV.  Transients  of  the  Adaptation  Processes 

A theory  for  the  transients  for  adaptation  processes  (bounds)  does  not 
yet  exist.  Very  few  references  on  this  area  are  available. 

1 M 

Some  work  has  been  done  to  determine  the  influence  of  the  various  design 
choices  upon  the  transients: 

- Choice  of  the  gain  sequence:  The  basic  adaptation  algorithms  are  of 
the  form: 


P (k+1 ) - p(k)  + FlA,k+1 

(1) 

P(k+1)  - p(k)  + 

(2) 

where  (1)  is  also  called  of  L.S.  type  (F^  is  a matrix  adaptation  gain),  and 
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(2)  Is  called  of  "stochastic  approximation"  type  (y^  - scalar  adaptation  gain). 
For  (1),  the  gain  can  be  updated  by  the  general  formula: 

• xi<k>Fk‘ + vk>v£ 

0 < Xx(k)  <1  ; 0 < \2(k)  <2  ; FQ  > 0 


and  the  transients  will  depend  upon  the  choice  of  the  X^(k)  and  X2(k).  For 
(2),  the  basic  formula  is  y^  « -j-,  however  modification  of  this  rule  at  the 
beginning  of  the  adaptation  is  highly  beneficial  (Instead  of  y^  such  that 
ky^  - 1,  for  all  k,  one  uses  another  sequence  y^  such  that  ky^  starts  at  1, 
grows  to  2 and  then  returns  to  1). 

- Choice  of  the  strictly  positive  real  transfer  functions. 

The  designs  require  that  a certain  transfer  function: 


H(z_1) 


DU'1) 

A(z_1) 


be  strictly  positive  real  where  A(z_1)  is  known  and  D(z-1)  should  be  cal- 
culated in  order  to  satisfy  the  S.P.R.  condition.  The  transients  will  depend 
on  the  particular  choice  of  D(z  *). 

- Proportional  + integral  adaptation:  Adding  proportional  adaptation 
when  possible,  the  transients  can  be  improved. 

The  study  of  the  optimization  of  the  gain  profile  and  of  the  S.P.R. 
choice  is  very  important  for  assuring  high  performances  for  the  adaptive 
control  schemes. 


V.  Explicit  and  Implicit  MRAC 

Several  terms  are  used  to  designate  these  schemes: 

• Explicit  MRAC  - Direct  MRAC. 

• Implicit  MRAC  • Indirect  MRAC. 


Implicit  MRAC  belongs  also  to  the  class  of  adaptive  control  systems 
which  use  as  an  intermediate  step  an  adaptive  predictor  estimator.  This  kind 
of  scheme  is  potentially  richer  than  explicit  MRAC.  However  stability  and 
convergence  results  are  available  only  for  those  schemes  which  are  equivalent 
to  explicit  MRAC. 

VI.  Assumptions  on  the  Leading  Coefficient  of  the  Plant  Transfer  Function 
The  plant  transfer  function  considered  is: 

gdp(s) 

Tp " * 

The  designs  presented  in  the  literature  can  handle  several  cases: 

a)  g is  known, 

b)  the  sign  of  g is  known  and  a bound  (inferior  and/or  superior), 

c)  g is  unknown  but  of  constant  sign. 

The  problems  which  should  be  considered  in  more  detail  are: 

• zero  crossing  during  adaptation  transients,  and 
- change  in  sign  of  g. 

The  most  difficult  problem  is  the  change  of  sign  of  g and  "cautious"  or 
"dual"  techniques  should  be  used  in  order  to  avoid  large  transients. 

VII.  Adaptive  Control  of  Non-Minimum  Phase  Plants 

Not  any  technique  fully  satisfactorily  for  handling  this  type  of  problem 
is  available.  The  available  techniques  are  basically  three. 

1.  Explicit  identification  of  the  zeros  and  polynomial  division.  Leads 
to  numerical  problems. 

2.  One  step  ahead  criterion  optimization  (Clarke,  Gathrop,  Johnson). 


Can  not  stabilize  any  plant 
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3.  Implicit  MRAC  using  an  adaptive  observer  in  "controllable  canonical 
form"  or  Explicit  MRAC  with  an  evolutive  reference  model  (Silveira).  The 
plant  should  be  stable  or  stabilizable  by  an  output  constant  feedback  for  all 
the  possible  values  of  plant  parameters. 

VIII.  Reduced  Order  Models 

The  reduced  order  models  enter  in  the  adaptive  control  problem  because 
the  plant  transfer  function  considered  in  the  design  is  either  an  approxi- 
mation done  because  a part  of  the  dynamics  can  be  neglected  in  open  loop  or 
one  would  like  to  have  a certain  controller  complexity. 

The  adaptive  control  scheme  features  two  time  scale  phenomena  allowing 
a separation  and  this  separation  can  be  parametrized  in  terms  of  a particular 
pi.  The  approximate  design  corresponds  to  p « 0 and  the  use  of  singular  per- 
turbation techniques  will  allow  to  measure  the  validity  of  this  approximation. 

The  need  for  implementation  of  the  transient  adaptation  signals  can  also 
be  probably  examined  within  the  same  context. 

IX.  Suggestions  for  Theoretical  Research 

The  subjects  which  are  listed  below  have  been  considered  not  only  as 
being  very  useful  for  the  development  of  MRAC-STURE  techniques  but  the  basis 
are  available  and  therefore  the  research  can  be  started  immediately. 

1.  Theoretical  analysis  of  adaptation  transients. 

2.  Analysis  under  disturbances. 

3.  Adaptive  control  of  non-minimum  phase  plant. 

4.  Stability  and  convergence  analysis  of  certain  known  schemes. 

5.  Parametrization  studies  for  control  of  M.I.M.O.  system  in  view  of 
the  use  of  adaptive  control. 

6.  Adaptive  control  of  M.I.M.O.  systems. 
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7.  Design  of  adaptive  control  schemes  for  restricted  domain  of  para- 
meter  variations. 

8.  Design  of  adaptive  control  schemes  using  adjustable  reference  models. 

9.  Effects  of  saturation  on  the  control  and  its  derivative. 

10.  Adaptive  control  of  time  varying  plants. 

11.  Adaptive  control  of  certain  classes  of  non-linear  plants. 

12.  Analysis  of  adaptive  control  schemes  using  reduced  order  models. 

13.  Use  of  singular  perturbation  methods  for  MRAC  analysis. 

14.  Robustness  of  adaptive  control  schemes  with  respect  to  design 
assumptions. 

15.  Loop  gain  and  frequency  response  interpretation  of  the  adaptive 

loop. 

16.  How  to  take  in  account  the  available  information  for  the  design  of 
adaptive  control  schemes. 

17.  Development  of  alternative  adaptation  schemes. 

18.  The  analysis  of  "subsystem1'  type  effects  upon  adaptive  control 
scheme  (ex:  adaptive  voltage  control  in  power  systems). 

The  order  of  the  various  research  subjects  does  not  represent  an  order  of 
preference.  However,  with  respect  to  the  potential  demand  of  users  for 
adaptive  control,  the  first  six  subjects  should  be  considered  with  a certain 
priority. 

X.  Suggestions  for  Applications 

The  discussions  were  oriented  essentially  with  respect  to  applications 
in  aeronautics  and  astronautics. 

It  appeared  that  the  application  of  MRAC-STURE  techniques  to  the  C |3  | 
variable  stability  aircraft  is  feasible  and  will  allow  an  extensive  evaluation 


under  various  conditions.  It  was  pointed  out  that  linear  model  following 


control  has  been  already  implemented  on  this  aircraft 


Other  possible  applications  Include:  space  shuttle,  missiles.  Jet 


engines,  pointer  systems,  submarines.  In  addition,  studies  concerning  the 
potential  use  of  MRAC-STURE  for  reconfiguration  of  control  after  fault  and 
for  high  angle  of  attack  aircraft  will  be  useful. 

Several  successful  applications  of  MRAC-STURE  techniques  for  the  control 


of  various  industrial  (or  pilot)  processes  have  been  done  in  the  last  S years 


and  mentioned  during  the  discussions.  However  these  techniques  have  not  yet 


become  standard  techniques  and  more  effort  should  be  done  in  this  direction 


Report  of  the 


WORKING  GROUP  ON  STOCHASTIC  ADAPTIVE  CONTROL 


Discussion  Leader:  Y.  Bar-Shalom 


1.  Role  of  Stochastic  Adaptive  Control 


Sources  of  uncertainty:  parameters  (“slowly"  varying  states),  process 


and  measurement  disturbance 


If  (1)  a loss  function  is  used  as  performance  measure  and  (2)  the  un 


certainties  are  modeled  probabilistically  then  only  Stochastic  Control 


methodology  can  evaluate  the  performance  degradation  due  to  each  (and  all) 


uncertainties — which  parameters  are  critical  to  control  performance  (A  vs.  B) 


Meaning  of  expected  value  of  loss  function 


minimum  variance  (1  step  horizon) 


general  - N step  horizon,  weighted  for  state  components  and  control 


The  stochastic  control  approach  yields  a control  law  that  depends  on 


(1)  current  information  (adaptivity) 


(2)  quality  of  the  current  information,  and 


(3)  anticipated  quality  of  future  information 


Other  approaches  do  not  incorporate  (2),  (3) 


Stochastic  control  methodology  can  be  used  to  characterize  conditions 


when  simpler  algorithms  are  adequate 


when  some  uncertainties  can  be  neglected 


when  the  results  of  an  identification  procedure  are  adequate  for  the 


control  purpose  (identification  accuracy  requirements  should  be  determined  by 


the  control  problem  where  the  model  is  used) 


In  the  overall  system  optimization 


the  estimation  has  to  serve  the  control  purpose 
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* the  control  has  in  general  a dual  purpose  (1)  control  proper  and  (2) 
estimation  enhancement. 

Therefore  simultaneous  optimization  is  needed. 

In  some  missile  guidance  problems  sophisticated  (but  not  necessarily 
expensive)  control  can  enhance  estimation  quality  or  reduce  sensor  require- 
ments . 

Such  guidance  can  yield  range  information  from  angle  only  sensors.  This 
(1)  reduces  missile  distance  and  (2)  allows  adaptive  estimation  (noise  co- 
variance  is  r-dependent). 

Adaptive  stochastic  control  can  be  used  to  guarantee  (with  probability  1) 
stability  of  an  unknown  system  with  disturbances  under  certain  conditions. 

This  is  an  alternate  approach  to  minimization  of  expected  loss. 


I 


2.  Existing  Stochastic  Control  Algorithms 
A.  Continuous  valued  parameters 


HCE  - uses  parameter  estimates  in  place  of  the  true  values.  If  un 


certainties  are  "small"  it  can  be  adequate.  Stochastic  analysis  is  required 


to  reach  this  conclusion  without  extensive  simulations 


STUBS  - one  step  look-ahead  minimum  variance.  Simplest,  and  extensively 


studied  implemented  for  process  control 


Modified  STURE  - to  include  also  one  step  ahead  estimation  enhancement 


SAFER  (sensitivity  adaptive  feedback  with  estimation  redistribution) 
exploits  sensitivity  to  improve  the  estimation  for  the  control  purpose. 

WSADC  • approximation  of  the  dynamic  programing  that  yields  explicit 


expressions  of  the  stochastic  effects  (divided  into  caution  and  probing  terms) 
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Min-max  increment  algorithm  - minimizes  the  largest  performance  de- 
gradation within  the  range  of  unknown  parameters. 

B.  Discrete  valued  parameters 

HCE  (heuristic  certainty  equivalence)  - uses  parameter  estimates  in 
place  of  the  true  values.  If  uncertainties  are  "small"  It  can  be  adequate. 
Stochastic  analysis  is  required  to  reach  this  conclusion  without  extensive 
simulations. 

MMAC  (partition)  algorithm  - model-optimum  controls  weighted  by  their 
a posteriori  probabilities. 

MAD  - model  adaptive  dual  - has  the  identification  enhancement  feature. 
3.  Topics  for  Future  Research 

• Nonlinear  stochastic  control  problems  - only  specific  classes  of 
problems  can  be  solved,  e.g.  parametric  imbedding  of  nonlinear  problems. 

• Convergence  (and  rate  of  convergence)  of  stochastic  adaptive  control 
algorithms  and  stability  of  the  overall  system. 

• Use  of  robust  structures  to  initialize  the  adaptation  process;  trade- 
off between  robust  structures  and  accuracy  needs  of  adaptation. 

• Development  of  meaningful  methodology  for  comparison  of  stochastic 
algorithms. 

• Interplay  of  model  choice  and  control  law. 

• Modeling  of  sources  of  uncertainty  (Markov,  others?). 

• Development  of  tools  for  non-Monte  Carlo  quantitative  assessment  of 
the  performance  improvement  obtainable  by  a stochastic  adaptive  control 
approach. 

• Development  of  actively  adaptive  (estimation  enhancing)  control 
algorithms  for  new  classes  of  problem  - missile  guidance,  ECM. 


Probing  - Caution  interrelationship 


Simplified  actively  adaptive  algorithms 


Relationship  among  the  various  existing  schemes  (Bayesian  vs.  min-max) 


V.  SUMMARY  AND  CONCLUSIONS 


The  adaptive  control  field  exists  because  of  a need  to  achieve  high 


system  performance  in  spite  of  severe  variations  in  the  characteristics  of 
the  process  to  be  controlled  and  in  spite  of  a vide  range  of  operating  and 


environmental  conditions.  Two  complementary  approaches  to  the  representation 


of  uncertainty,  deterministic  and  stochastic,  are  useful.  Both  character 


izations  are  employed  in  the  investigation  of  fixed  feedback  structures  and 


the  investigation  of  adaptive  controllers  which  automatically  adjust  to 


changes  in  the  process  characteristics.  Fixed  controllers  which  Insure 


satisfactory  operation  in  spite  of  vide  variations  in  process  characteristics 


are  called  robust  controllers.  The  most  common  adaptive  controllers  are  the 


model-reference  adaptive  controller  and  the  self-tuning  regulator.  The  most 


demanding  adaptive  control  problem  is  stochastic  with  a general  performance 


criterion  containing  several  stages 


Whenever  robust  controls  can  perform  satisfactorily  in  solving  adaptive 


control  problems,  they  offer  an  attractive  alternative  to  active  adaptive 


controls  because  of  their  relative  simplicity  in  realization  and  hence  greater 


reliability.  Even  when  adaptive  controls  are  to  be  used,  the  robust 


controller  is  a useful  backup  to  provide  adequate  control  for  emergency 


conditions.  Furthermore,  adaptive  controllers  which  are  also  robust  would  be 


desirable.  Concepts  from  robust  control  theory  would  enrich  the  field  of 


active  adaptive  control.  Several  Important  conclusions  regarding  robust 


controls  are  listed  in  Section  IV-A. 


Model  reference  adaptive  controls  have  been  investigated  for  almost 


twenty  years  now  and  many  results  are  ready  for  applications.  Many  of  the 


recent  results  are  listed  in  the  references  of  Section  IXX-B.  One  important 
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recent  result  concerns  stability  and  convergence.  Because  o£  the  focus  on 
this  well-defined  problem  area,  several  specific  suggested  areas  for  future 
research  resulted  from  the  discussion  group.  These  are  listed  in  Section 
TV-B.  Several  of  these  could  be  expected  to  be  solved  in  the  near  future. 

. In  the  area  of  general  stochastic  adaptive  control,  the  problem  is  very 

difficult  and  most  results  to  date  are  quite  Involved.  An  important  exception 

% 

is  the  self-tuning  regulator  which  is  developed  largely  for  single-input 
single-output  linear  systems  with  minimization  of  a one-step  ahead  mean- 
square  error.  This  has  been  developed  over  the  last  twelve  years  and  it  has 
been  successfully  applied  to  industrial  process  control  problems.  Other 
special  classes  of  stochastic  adaptive  control  problems  are  under  in- 
vestigation and  the  researchers  in  the  field  are  seeking  practical  control 
algorithms.  Several  important  application  problems  were  discussed  in  the 
working  group  and  several  suggested  directions  for  research  are  listed  in 
Section  III-C. 

From  the  discussions  of  all  three  working  groups,  it  was  clear  that  con- 
cepts of  robust  control  could  significantly  enrich  the  study  of  adaptive 
control.  Adaptive  controllers  should  have  some  robustness  properties  so  that 
the  adaptation  would  not  have  to  be  too  critical.  In  the  context  of 
stochastic  control,  the  estimation  aspect  of  the  control  can  be  simpler  if 
the  control  is  robust.  Moreover,  concepts  of  sensitivity  which  are  deeply 
rooted  in  feedback  theory  can  be  exploited  in  stochastic  controllers  with 
dual  effects. 

The  size  of  the  workshop  and  the  particular  mix  among  industrial  repre- 
sentatives, academic  researchers,  and  government  laboratory  scientists  and 
engineers  appeared  to  be  just  right.  The  Informal  arrangement  and  organ- 

I 

ization  encouraged  much  discussion,  interchange  of  ideas,  and  stimulation. 
There  was  much  enthusiasm  for  further  research  in  adaptive  control. 
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APPENDIX  A 


AFOSR  WORKSHOP  ON  ADAPTIVE  CONTROL 

University  Inn,  Champaign,  Illinois 
May  8-10,  1979 


Plaza  Room,  8:30  a.m.-5:00  p.m 


Registration 

Opening  Remarks,  Major  C.  L.  Nefzger,  AFOSR 
Overview  of  Model  Reference  Adaptive  Control  and  Self- 
Tuning  Regulators,  I.  D.  Landau,  Laboratoire 
d'Automatique  de  Grenoble,  ENSIEG,  France 
Coffee 

Overview  of  Stochastic  Adaptive  Control,  Y.  Bar-Shalom, 
University  of  Connecticut 

Overview  of  Robust  Control,  J.  Ackermann,  Institute  for 
Dynamics  of  Flight  Systems,  DFVLR,  Germany 
General  Discussion 
Coffee 

Geaeral  Discussion 
Organization  of  Working  Groups 

Model  Reference  Adaptive  Control,  Plaza  Room  A 
Stochastic  Adaptive  Control,  Plaza  Room  B 

Robust  Control,  Plaza  Room  C 


Working  Group  on  Model  Reference  Adaptive  Control  and  Self-Tuning 
Regulators,  I.  D.  Landau,  Discussion  Leader 
Working  Group  on  Stochastic  Adaptive  Control,  Y.  Bar-Shalom,  Discussion 
Leader 

Working  Group  on  Robust  Control,  J.  Ackermann,  Discussion  Leader 


These  groups  will  meet  in  Plaza  Rooms  A,  B,  and  C.  The  meeting  rooms 
will  be  open  8:30  a.m.  - 5:00  p.m.  Upon  request  they  may  be  open  in 
the  evenings  for  extended  discussions. 


Plaza  Room,  8:00  a.m 


8:00  a.m.  Report  of  Working  Group  A,  I.  D.  Landau 

8:30  a.m.  Report  of  Working  Group  B,  Y.  Bar-Shalom 

9:00  a.m.  Report  of  Working  Group  C,  J.  Ackermann 

9:30-  General  Discussion  and  Concluding  Statements 

10:30  a.m. 

10:45  a.m.  Workshop  ends. 
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Champaign,  Illinois,  May  8-10,  1979 
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Dept,  of  Electrical  Engineering 
and  Computer  Science 
University  of  Connecticut 
Storrs,  Connecticut  06268 

Mr.  David  Bowser 
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Control  Analysis  Group 

Flight  Control  Division 

Air  Force  Flight  Dynamics  Lab. 

Wright -Patterson  Air  Force  Base, 

Ohio  45433 

Professor  Peter  E.  Caines 
Pierce  Hall 
Harvard  University 
Cambridge,  Massachusetts  02138 

Professor  J.  B.  Cruz,  Jr. 

Decision  and  Control  Laboratory 
Coordinated  Science  Laboratory 
University  of  Illinois 
Urbana,  Illinois  61801 

Lt.  Colonel  James  Dillow 

Air  Force  Weapons  Laboratory/ ALO 

Kirtland  Air  Force  Base, 

New  Mexico  87117 

Professor  Gene  F.  Franklin 
Information  Systems  Laboratory 
Department  of  Electrical  Engineering 
Stanford  University 
Stanford,  California  94305 


Dr.  Bernard  Friedland 
Aerospace  & Marine  Systems 
Kearfott  Division,  SINGER 
1150  McBride  Avenue 
Little  Falls,  New  Jersey  07424 

Dr.  C.  A.  Harvey 
Honeywell  Systems  and  Research 
Center 

Aerospace  and  Defense  Group 
2600  Ridge  Parkway 
Minneapolis,  Minnesota  55413 

Professor  C.  Richard  Johnson,  Jr. 
Department  of  Electrical  Engineering 
Virginia  Polytechnic  Institute  and 
State  University 
Blacksburg,  Virginia  24061 

Professor  Howard  Kaufman 
Dept,  of  Electrical  & Systems 
Engineering 

Rensselaer  Polytechnic  Institute 
Troy,  New  York  12181 

Professor  Petar  V.  Kokotovic 
Decision  and  Control  Laboratory 
Coordinated  Science  Laboratory 
University  of  Illinois 
Urbana,  Illinois  61801 

Dr.  G.  Kreisselmeier 
Institute  for  Dynamics  of  Flight 
Systems 
DFVLR 

Oberpfaffenhofen,  8031 
Wessling,  F.R.  Germany 

Professor  D.  G.  Lainiotis 
Department  of  Electrical  Engineering 
State  University  of  New  York  at 
Buffalo 

Amherst,  New  York  14260 
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Or.  I.  D.  Landau 
Laboratoire  de’Automatique  de 
Grenoble  (CNRS) 

ENSIEG 

38402  Saint  Martin  D'Heres 
France 

Professor  Douglas  P.  Loose 
Decision  & Control  Laboratory 
Coordinated  Science  Laboratory 
University  of  Illinois 
Urbana,  Illinois  61801 

Professor  Lennart  Ljung 

Department  of  Electrical  Engineering 

Linkoping  University 

S-581  83  Linkoping 

Sweden 

Mr.  Richard  C.  Marsh 
Department  338 

McDonnell  Douglas  Corporation 

P.  0.  Box  516 

St.  Louis,  Missouri  63166 

Dr.’  Juraj  Medanic 
Mijailo  Pup in  Institute 
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Dr.  Raman  K.  Mehra 
Scientific  Systems,  Inc. 

Suite  309-310 
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Professor  Richard  V.  Monopoli 
Department  of  Electrical  and 
Computer  Engineering 
University  of  Massachusetts 
Amherst,  Massachusetts  01002 

Professor  A.  S.  Morse 
Dept,  of  Engineering  and  Applied 
Science 

Yale  University 

New  Haven,  Connecticut  06520 


Professor  K.  S.  Narendra 
Dept,  of  Engineering  and  Applied 
Science 

Yale  University 

New  Haven,  Connecticut  06520 

Major  Charles  L.  Nefzger 
Air  Force  Office  of  Scientific 
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Directorate  of  Mathematical  and 
Information  Sciences  (Ml) 

Building  410 

Bolling  Air  Force  Base,  D.C.  20332 

Dra.  Consuelo  S.  Padilla 
Systems  Engineering  Laboratory 
IVIC  - Ingenieria  II 
Apartado  1827 
Caracas,  Venezuela 
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Decision  and  Control  Laboratory 
Coordinated  Science  Laboratory 
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Urbana,  Illinois  61801 
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Dept,  of  Electrical  Engineering- 
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Los  Angeles,  California  90007 

Professor  Anthony  V.  Sebald 
Dept,  of  Applied  Mechanics  and 
Engineering  Sciences 
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University  of  Southern  California 
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